Abstract This document defines the data collection methodology for the EMC modeling tools. The target audience is composed of EMC GS/TS Technology Consultants and TS Solutions Architects. EMC Modeling Tools Data Collection Guide EMC GS/TIS P2/68 Technology Implementation Services
Copyright 2007, 2013 EMC Corporation. All rights reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS." EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC2, EMC, EMC ControlCenter, AlphaStor, ApplicationXtender, Captiva, Catalog Solution, Celerra, CentraStar, CLARalert, CLARiiON, ClientPak, Connectrix, Co- StandbyServer, Dantz, Direct Matrix Architecture, DiskXtender, DiskXtender 2000, Documentum, EmailXaminer, EmailXtender, EmailXtract, eRoom, FLARE, HighRoad, Input Accel, Navisphere, NetWorker, nLayers, OpenScale, Powerlink, PowerPath, Rainfinity, RepliStor, ResourcePak, Retrospect, Smarts, SnapShotServer, SnapView/IP, SRDF, Symmetrix, TimeFinder, VisualSAN, VSAM-Assist, WebXtender, where information lives, Xtender, and Xtender Solutions are registered trademarks and EMC Developers Program, EMC OnCourse, EMC Proven, EMC Snap, EMC Storage Administrator, Acartus, Access Logix, ArchiveXtender, Authentic Problems, Automated Resource Manager, AutoStart, AutoSwap, AVALONidm, C-Clip, Celerra Replicator, Centera, CLARevent, Codebook Correlation Technology, Common Information Model, CopyCross, CopyPoint, DatabaseXtender, Direct Matrix, EDM, E-Lab, Enginuity, FarPoint, Global File Virtualization, Graphic Visualization, InfoMover, Infoscape, Invista, Max Retriever, MediaStor, MirrorView, NetWin, OnAlert, PowerSnap, RepliCare, SafeLine, SAN Advisor, SAN Copy, SAN Manager, SDMS, SnapImage, SnapSure, SnapView, StorageScope, SupportMate, SymmAPI, SymmEnabler, Symmetrix DMX, UltraPoint, UltraScale, Viewlets, and VisualSRM are trademarks of EMC Corporation. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com All other trademarks used herein are the property of their respective owners. EMC Modeling Tools Data Collection Guide EMC GS/TIS P3/68 Technology Implementation Services
TABLE OF CONTENTS Document Control ................................................................................................................................................ 7 Contributors ...................................................................................................................................................................... 7 Build Status ...................................................................................................................................................................... 7 1 Introduction to Data Collection ..................................................................................................................... 9 1.1 Recommended 3 Day Minimum at 10 Minute Intervals ............................................................................................ 9 1.2 Minimum Data Collection Interval is 1 Minute......................................................................................................... 9 1.3 Data Collection Interval Must Be the Same............................................................................................................. 9 1.4 Standardized Statistics ......................................................................................................................................... 9 1.4.1 Required Fields ........................................................................................................................................ 9 1.4.2 Optional Fields ........................................................................................................................................ 9 1.5 Supported Array and Host Statistics ..................................................................................................................... 10 1.6 General Assumptions ........................................................................................................................................... 10 1.7 Sample Data ........................................................................................................................................................ 11 1.7.1 EMC Internal Users .................................................................................................................................. 11 1.7.2 External Users......................................................................................................................................... 11 2 AIX iostat .................................................................................................................................................... 12 2.1 Data Collection Considerations ............................................................................................................................ 12 2.2 Data Collection Procedure .................................................................................................................................... 12 2.3 Command Syntax ................................................................................................................................................. 12 2.4 Record Layout ...................................................................................................................................................... 13 2.5 Sample Output .................................................................................................................................................... 13 3 BCSD Export ................................................................................................................................................ 14 3.1 Data Collection Procedure .................................................................................................................................... 14 3.2 Record Layout ...................................................................................................................................................... 14 3.3 Sample Output .................................................................................................................................................... 14 4 BTP ............................................................................................................................................................. 15 4.1 Collection Considerations .................................................................................................................................... 15 4.1.1 RAID-10 ................................................................................................................................................. 15 EMC Modeling Tools Data Collection Guide EMC GS/TIS P4/68 Technology Implementation Services
4.1.2 Time Zone .............................................................................................................................................. 15 4.2 Data Collection Procedure .................................................................................................................................... 15 4.2.1 Performance Manager ............................................................................................................................ 15 4.2.2 STP ........................................................................................................................................................ 16 5 HP-UX sar .................................................................................................................................................... 17 5.1 Data Collection Considerations ............................................................................................................................ 17 5.2 Data Collection Procedure .................................................................................................................................... 17 5.2.1 Command Syntax ................................................................................................................................... 18 5.3 Record Layout ...................................................................................................................................................... 19 5.4 Sample Output ................................................................................................................................................... 20 6 iSeries (System i, IBM i) WRKDSKSTS .......................................................................................................... 21 6.1 Data Collection Considerations ............................................................................................................................ 21 6.2 Data Collection Procedure .................................................................................................................................... 21 6.3 Command Syntax ................................................................................................................................................. 21 6.4 Record Layout ..................................................................................................................................................... 22 6.5 Sample Output ................................................................................................................................................... 24 7 Linux iostat ................................................................................................................................................ 25 7.1 Data Collection Considerations ............................................................................................................................ 25 7.2 Data Collection Procedure .................................................................................................................................... 25 7.3 Command Syntax ................................................................................................................................................. 25 7.4 Record Layout ..................................................................................................................................................... 26 7.5 Sample Output ................................................................................................................................................... 28 8 Mainframe (SMF/CMF) ................................................................................................................................ 29 8.1 Data Collection Considerations ........................................................................................................................... 29 8.2 Mainframe Assembler Kit .................................................................................................................................... 30 8.2.1 Additional notes .................................................................................................................................... 30 8.3 Onsite Processing ................................................................................................................................................ 31 8.4 Offsite Processing ................................................................................................................................................ 31 8.5 Record Layout ...................................................................................................................................................... 33 8.6 Sample Output .................................................................................................................................................... 35 EMC Modeling Tools Data Collection Guide EMC GS/TIS P5/68 Technology Implementation Services
8.7 Accessing EMCs E01 LPAR .................................................................................................................................. 36 9 MirrorView/A LOR (locality of reference)..................................................................................................... 39 9.1 Data Collection Considerations ........................................................................................................................... 39 9.2 Acquiring the MirrorView/A LOR Tool ................................................................................................................... 39 9.3 Data Collection Procedure ................................................................................................................................... 39 9.4 Command Syntax ................................................................................................................................................ 39 9.5 Sample Output ................................................................................................................................................... 40 10 Navisphere Analyzer (NAR) .......................................................................................................................... 41 10.1 Data Collection Considerations for metaLUNs and Thin LUNs ................................................................................. 41 10.1.1 RAW files ............................................................................................................................................... 41 10.1.2 CSV files ................................................................................................................................................ 41 10.2 Data Collection Procedure .................................................................................................................................... 41 10.2.2 Sample batch file to convert all NAR files in a directory to CSV files ......................................................... 42 10.3 Command Syntax ................................................................................................................................................ 42 10.4 Record Layout ......................................................................................................................................................44 10.5 Sample Output .................................................................................................................................................... 45 11 Solaris iostat ............................................................................................................................................. 46 11.1 Data Collection Considerations ........................................................................................................................... 46 11.2 Data Collection Procedure ................................................................................................................................... 46 11.3 Command Syntax ................................................................................................................................................ 46 11.4 Record Layout ...................................................................................................................................................... 47 11.5 Sample Output .................................................................................................................................................... 47 12 VMware esxtop/resxtop ............................................................................................................................. 48 12.1 Data Collection Considerations ........................................................................................................................... 48 12.2 Data Collection Procedure ................................................................................................................................... 48 12.3 Command Syntax ................................................................................................................................................ 49 12.4 Record Layout ..................................................................................................................................................... 49 12.5 Sample Output ....................................................................................................................................................50 13 VPLEX Stats ................................................................................................................................................. 51 13.1 Data Collection Considerations ............................................................................................................................ 51 EMC Modeling Tools Data Collection Guide EMC GS/TIS P6/68 Technology Implementation Services
13.2 Data Collection Procedure .................................................................................................................................... 51 13.3 Command Syntax ................................................................................................................................................. 51 13.4 Record Layout ...................................................................................................................................................... 52 13.5 Sample Output .................................................................................................................................................... 53 14 Windows Perfmon ...................................................................................................................................... 54 14.1 Data Collection Considerations ............................................................................................................................ 54 14.2 Data Collection Procedure .................................................................................................................................... 54 14.3 Conversion from BLG to CSV ................................................................................................................................ 62 14.4 Record Layout ..................................................................................................................................................... 62 14.5 Sample Filtered Output ....................................................................................................................................... 64 Appendix A TroubleShooting and Support ...................................................................................................... 65 Appendix B Feature Enhancements and Feedback ........................................................................................... 66 B.1 Feature Enhancements .............................................................................................................................................. 66 B.2 Feedback .................................................................................................................................................................. 68
DOCUMENT CONTROL Contributors Role Name Author(s) Mike Lawrence Build Status Version Date of issue Author Reason 1.10 10-Mar-2011 Mike Lawrence 1. Rewrite data collection procedure for VMware esxtop. 2. FLARE 30 requires a change to the collection procedure for NAR CSV. 1.11 13-Dec-2011 Mike Lawrence 1. Clarify how to use metaLUNs when using NAR files and include/exclude lists. 2. Removed NaviCLI commands for pre- FLARE 24. 3. Add VPLEX stats support 4. As the SVC no longer exists, removed the section on the SVC. 1.12 30-May-2012 Mike Lawrence Correct some typos. 1.13 25-Nov-2012 Mike Lawrence 1. Change NAR section to include raw. 2. Change - archivedump to- archivedump (removed the space). 3. For all data imported into the same application, the data collection interval must be the same. 4. Change WRKDSKSTS total read and write IOs to IOPS. 1.14 29-Jan-2013 Mike Lawrence Dropped CLARiiON/Unified NAR CSV and Simulated (Workload Generator) performance data. 1.15 29-May-2013 Mike Lawrence 1. Updated Windows Perfmon data collection for Windows 2008 and above. 2. Added Linux iostat considerations for partitioned devices and emcpower devices. 3. Removed references to the now retired ET tool. 4. Added T option as default parameter for AIX iostat as newer versions of the OS support it. 5. Added the collection consideration, the read/write ratio is more of a "best guess" based on all of the read/write activity for all of the volumes in any given interval. In other words, it is not accurate, for HP-UX sar. 6. Added CLARiiON/Unified NAR CSV as it is temporarily back in BCSD. 7. Added to HP-UX sar Data Collection Considerations. 8. Added to Minimum Data Collection EMC Modeling Tools Data Collection Guide EMC GS/TIS P8/68 Technology Implementation Services
Interval is 1 Minute. 1.16 03-Jul-2013 Mike Lawrence 1. Correct Windows Perfmon relog syntax. 2. Added consideration for AIX iostat partitioned records. 1.17 14-Nov-2013 Mike Lawrence Removed comments about iSeries RAID-1 volumes as the parser now correctly handles them.
1 INTRODUCTION TO DATA COLLECTION EMCs remote replication modeling tools require host or array based statistical data to determine data change activity. This enables modeling based on real world data rather than a best guess based on highly averaged data. 1.1 Recommended 3 Day Minimum at 10 Minute Intervals The minimum data capture interval should be 3 days, while a week is quite common as it will include the various daily peaks and valleys. The actual times for the peaks should be the responsibility of the customer as they know when their peak processing occurs. Ideally, the data capture will include the peak workload period such as month-end or year-end processing. By capturing at least 24 hours, a distinct online and batch processing profile is observed. This is important because the average IO size of write I/Os tends to be smaller for online workloads and larger for batch, which has a direct impact on bandwidth requirements. Data capture should be in 10 minute intervals or less, but no less than 1 minute. If longer intervals are used you may be impacted by the flattening out of the true peaks due to longer averages, thus skewing the results. Even at 10 minute intervals your true peaks could be twice as much as the average peak recorded in the interval. This can cause a significantly under-configured solution. If a collection interval of less than 10 minutes is desired, note the tools require a minimum of 1 minute. 1.2 Minimum Data Collection Interval is 1 Minute The minimum data collection interval is 1 minute. Using a data collection interval of less than 1 minute will result in unpredictable results such as corrupted data, invalid error messages, and invalid modeling results. This is due to the fact that the time rounding routine rounds to the nearest data collection interval in minutes. 1.3 Data Collection Interval Must Be the Same For all data imported into the same application, the data collection interval must be the same. 1.4 Standardized Statistics Performance data is imported into the tool where all statistics are normalized into a single format. 1.4.1 Required Fields Field Description Timestamp Date and Time of the performance data (converted to GMT based on user specified offset). The timestamp is rounded to the nearest collection interval. Read I/Os Total read I/Os Read I/O Size Average read I/O size (bytes) Write I/Os Total write I/Os Sequential WriteIOs Total sequential write I/Os (If not contained in the data, set to zero) Write IO Size Average write I/O size (bytes) Volume ID LUN, device name, volume ID or volser 1.4.2 Optional Fields These fields are used exclusively for BCSDs VPLEX modeling. Field Description RRH I/Os Random read hit IO type EMC Modeling Tools Data Collection Guide EMC GS/TIS P10/68 Technology Implementation Services
Field Description RRM I/Os Random read miss IO type SR I/Os Sequential read IO type RW I/Os Random write IO type SW I/Os Sequential write IO type
Statistics Description AIX iostat IBM AIX iostat command output Symmetrix BTP EMC Symmetrix data in binary format HP-UX sar HP-UX sar command output iSeries WRKDSKSTATS IBM iSeries WRKDSKSTATS Linux iostat Linux iostat command output Mainframe Assembler Kit v2 IBM mainframe SMF data processed by EMCs Mainframe Assembler Kit v2 and output in CSV format MirrorView/A LOR Tool EMCs MirrorView/A LOR tool output in CSV format CLARiiON/Unified Block NAR CSV CLARiiON and Celerra Navisphere Analyzer data in CSV format CLARiiON/Unified Block NAR (raw) CLARiiON and Celerra Navisphere Analyzer data in raw format Solaris iostat Sun Solaris iostat command output Windows Perfmon CSV Microsoft Windows Perfmon data in CSV format (raw BLG data is not supported) VMware esxtop VMware esxtop command output (BCSD only) VPLEX stats VPLEX statistics output (BCSD only) 1.6 General Assumptions All headings are in English, so non-English headings need to be translated to English. Missing data such as timestamps and collection intervals will be supplied by the user during importing. For BCSD, after determining the IO rates and/or counts: o All records with zero I/O's for reads, writes and sequential writes will be skipped and not imported. o All reads and writes with more than 250,000 IOPS will be flagged and excluded from modeling. o Any write IO size larger than 20MB will be flagged and excluded from modeling. EMC Modeling Tools Data Collection Guide EMC GS/TIS P11/68 Technology Implementation Services
1.7 Sample Data 1.7.1 EMC Internal Users Sample data can be found in the EMC GS Tools eRoom in the GS Tools Common Components folder at http://ctseroom02.corp.emc.com/eRoom/tsfuncprac/TSTools/0_47cc6. Internal EMC users who do not have access may request it by emailing TS_Tools_Support@emc.com. but all other users must request copies of the data. 1.7.2 External Users You may request copies of the sample data by emailing TS_Tools_Support@emc.com. EMC Modeling Tools Data Collection Guide EMC GS/TIS P12/68 Technology Implementation Services
2 AIX IOSTAT 2.1 Data Collection Considerations AIX iostat formats may vary from version to version with some versions providing timestamps while others do not. If timestamps are not present then you will need to know when the data was collected and what collection interval was used and enter it on import. Partitioned records are redundant data and will be ignored. sd* device driver names (as an example, sddj and then sddj1) may possibly be multiple entries for the same basic host device. In the example previously, sddj will be totally encompassing for all subcomponent partitions (i.e. sddj will show total IOPs for sddj1 and sddj2 and any other partitions). Because of this, you will need an include list to strip out all numbers after the primary device driver name. The command can be scheduled in cron or other scheduling products for the desired capture period. 2.2 Data Collection Procedure Follow these steps to collect iostat data: 1. Identify the servers and the specific LUNS (disks) that will be replicated. 2. Run iostat for a short duration and verify the output matches the required format (see Record Layout and Sample Output below). 3. On each host, run iostat from the command line or optionally create a cron entry. 4. Collect the iostat output file(s). 2.3 Command Syntax The command syntax should be similar to the following and be issued on each host: iostat d T interval count >> hostname.iostat.date.txt Options Description -d Displays only the device utilization report -T Adds a column containing the time, but no date (AIX 5.2 and above) interval Collects data every specified seconds Suggested setting is 600 (10 minutes) count Collects statistics for specified intervals Suggested setting is 432 (3 days of 10 minute intervals) CRON Syntax: * * * * * iostat d T interval count >> hostname.iostat.date.txt - - - - - | | | | | | | | | + day of week ( 1 - 7) ( Monday = 1) ( Sunday = 0 or 7) | | | +- - - - - - - mont h ( 1 - 12) | | +- - - - - - - - - day of mont h ( 1 - 31) | +- - - - - - - - - - - hour ( 0 - 23) +- - - - - - - - - - - - - mi n ( 0 - 59) Sample CRON Entry: 0 0 * * 5-7 iostat d T 600 144 >> zeus.iostat.04142009.txt EMC Modeling Tools Data Collection Guide EMC GS/TIS P13/68 Technology Implementation Services
This entry schedules iostat to start Friday at 00:00 and end Sunday at 00:00. The iostat command pulls statistics every 600 seconds (10 minute) for 24 hours creating 144 intervals. The UNIX redirect >> appends the output to the same file. 2.4 Record Layout Heading Description Disks: Disk name % tm_act Percentage of time device was busy servicing a request Kbps Amount of data transferred (both reads and writes) to the drive in KB per second tps Total I/Os per second Kb_read Total amount of data read from the drive in KB Kb_wrtn Total amount of data written to the drive in KB time Time the statistics were written (5.2+) 2.5 Sample Output Without Time Syst emconf i gur at i on: l cpu=32 dr i ves=976 pat hs=2 vdi sks=0
3 BCSD EXPORT 3.1 Data Collection Procedure Data can be exported from BCSD by going to the replication/consistency/volume group and using the export performance data to CSV feature. This will export only the mapped data for that group. 3.2 Record Layout Field Description Performance Data Tag Project level user tag used to identify a group of performance data that will be accessed by one or more BCSD projects Load Tag Performance data level user tag used to identify specific performance data files File Name Name of the performance data file for these specific records Timestamp Date and Time of the loaded data after being normalized to GMT based on the user selected offset and rounded to the nearest collection interval Data Type Type of performance data Seconds per Interval Calculated time interval in seconds between two statistic records Volume ID LUN, device name, volume ID or volser Read IOs Total read IOs Read IO Size Average read IO size in bytes Write IOs Total write I/Os Sequential Write IOs Total sequential write I/Os (If not contained in the data, set to zero) Write IO Size Average write IO size in bytes Response Time Host IO response time in ms (If not contained in the data, set to zero) Random Read Hit Total random read hit IOPS Random Read Miss Total random read miss IOPS Sequential Read Total sequential read IOPS Random Write Total random write IOPS Sequential Write Total sequential write IOPS 3.3 Sample Output Per f or mance Dat a Tag, Load Tag, Fi l e Name, Ti mest amp, Dat a Type, Seconds per I nt er val , Vol ume I D, Read I Os, Read I O Si ze, Wr i t e I Os, Sequent i al Wr i t e I Os, Wr i t e I O Si ze, Response Ti me, RandomRead Hi t , RandomRead Mi ss, Sequent i al Read, RandomWr i t e, Sequent i al Wr i t e, Test , PeakFi l es, T1_20120414_180004_000292600925. bt p, 2012- 04- 14 18: 25: 00, BTP, 300, 027A, 124. 584716796875, 999. 4239501953125, 2. 9900331497192383, 0. 9966776967048645, 58 02. 66650390625, 0. 0, 0. 37541529536247253, 0. 039867103099823, 0. 0, 0. 006644518114626408, 0. 0033222590573 13204 Test , PeakFi l es, T1_20120414_180004_000292600925. bt p, 2012- 04- 14 19: 45: 00, BTP, 300, 027A, 64. 21404266357422, 880. 0, 0. 0, 0. 0, 0. 0, 0. 0, 0. 1872909665107727, 0. 02675585448741 913, 0. 0, 0. 0, 0. 0 Test , PeakFi l es, T1_20120414_180004_000292600925. bt p, 2012- 04- 14 20: 05: 00, BTP, 300, 027A, 67. 60562896728516, 944. 0, 0. 0, 0. 0, 0. 0, 0. 0, 0. 19718310236930847, 0. 0281690061092 3767, 0. 0, 0. 0, 0. 0 Test , PeakFi l es, T1_20120414_180004_000292600925. bt p, 2012- 04- 14 21: 05: 00, BTP, 300, 027A, 64. 21404266357422, 816. 0, 0. 0, 0. 0, 0. 0, 0. 0, 0. 18394649028778076, 0. 0301003307104 11072, 0. 0, 0. 0, 0. 0 EMC Modeling Tools Data Collection Guide EMC GS/TIS P15/68 Technology Implementation Services
Test , PeakFi l es, T1_20120414_180004_000292600925. bt p, 2012- 04- 14 21: 25: 00, BTP, 300, 027A, 64. 0, 1024. 0, 0. 0, 0. 0, 0. 0, 0. 0, 0. 19333332777023315, 0. 02000001072883606, 0. 0, 0. 0 , 0. 0 Test , PeakFi l es, T1_20120414_180004_000292600925. bt p, 2012- 04- 14 21: 45: 00, BTP, 300, 027A, 64. 21404266357422, 720. 0, 0. 0, 0. 0, 0. 0, 0. 0, 0. 20066890120506287, 0. 0133779197931 28967, 0. 0, 0. 0, 0. 0 Test , PeakFi l es, T1_20120414_180004_000292600925. bt p, 2012- 04- 14 18: 05: 00, BTP, 300, 027B, 0. 0, 0. 0, 64. 21404266357422, 0. 0, 512. 0, 0. 0, 0. 0, 0. 0, 0. 0, 0. 21404682099819183, 0. 0 Test , PeakFi l es, T1_20120414_180004_000292600925. bt p, 2012- 04- 14 18: 10: 00, BTP, 300, 027B, 0. 0, 0. 0, 128. 0, 0. 0, 512. 0, 0. 0, 0. 0, 0. 0, 0. 0, 0. 4266666769981384, 0. 0 Test , PeakFi l es, T1_20120414_180004_000292600925. bt p, 2012- 04- 14 18: 15: 00, BTP, 300, 027B, 0. 0, 0. 0, 64. 0, 0. 0, 512. 0, 0. 0, 0. 0, 0. 0, 0. 0, 0. 2133333384990692, 0. 0 Test , PeakFi l es, T1_20120414_180004_000292600925. bt p, 2012- 04- 14 18: 20: 00, BTP, 300, 027B, 0. 0, 0. 0, 128. 0, 0. 0, 680. 0, 0. 0, 0. 0, 0. 0, 0. 0, 0. 4266666769981384, 0. 0 4 BTP 4.1 Collection Considerations If your data will include metas, then you will need to specify both the meta-heads and meta-members for any include or exclude list used during modeling. Access to the Symmetrix BIN file or symapi_db.bin file is needed to accurately identify the meta-members. Use the Symmetrix Meta Member Expansion & Validation tool to expand your list of meta-heads for use as an include or exclude list. 4.1.1 RAID-10 A fix has been implemented in 5671, 5772, 5773 and 5874 for the issue where the RAID-10 volume statistics are written in total to all four meta-members, thus if processed as is would result in a 4x increase in the actual number of IOs. For additional information, see Primus EMC174047. When importing data containing RAID-10 volumes the EMC remote replication modeling tools scan through all of the imported records (during modeling) and for each RAID-10 group with the same read and write statistics, will divide those read and write statistics by 4. This is similar to the fix provided in SymmMerge. 4.1.2 Time Zone BCSD does not request a GMT offset and will always reflect GMT with zero offset, unless you specify a different offset at modeling time. There are several factors that can impact the accuracy of the BTP time-zone information when viewed by any of the EMC tools: The actual time-zone set on the Symmetrix/DMX service processor. The actual time-zone of the Agent used to collect the BTP data. The actual time-zone of where the data is being viewed. 4.2 Data Collection Procedure Follow these steps to collect BTP data: 1. Identify the arrays and the specific LUNS (disks) that will be replicated 2. For each array, use Performance Manager or STP to start BTP data collection 3. Collect the BTP or TTP output file(s) 4. For TTP files, convert them to BTP files using STPNavigator 4.2.1 Performance Manager To create BTP files, follow the same collection process as defined in ECC. As a recommendation, use the Analyst Method as it creates a single BTP file for any interval and duration specified by the user. EMC Modeling Tools Data Collection Guide EMC GS/TIS P16/68 Technology Implementation Services
4.2.2 STP Coordinate STP data collection with your Customer Engineer (CE) as they will need to setup a script to run on the Service Processor. EMC Modeling Tools Data Collection Guide EMC GS/TIS P17/68 Technology Implementation Services
5 HP-UX SAR 5.1 Data Collection Considerations The read/write ratio is a "best guess" based on all of the read/write activity for all of the volumes in any given interval. In other words, it is not accurate. If you can collect and use data from another source, that will be more accurate. The command can be scheduled in cron or other scheduling products for the desired capture period. 5.2 Data Collection Procedure Follow these steps to collect sar data: 1. Identify the servers and the specific LUNS (disks) that will be replicated. 2. Run sar for a short duration and verify the output matches the required format (see EMC Modeling Tools Data Collection Guide EMC GS/TIS P18/68 Technology Implementation Services
3. Record Layout and Sample Output below). 4. On each host, run sar from the command line or optionally create a cron entry. 5. Collect the sar output file(s). 5.2.1 Command Syntax The command syntax should be similar to the following and be issued on each host: sar d b interval count >> hostname.sar.date.txt
-d Displays the device activity -b Displays buffer I/O statistics interval Collects data every specified seconds Suggested setting is 600 (10 minutes) count Collects statistics for specified intervals Suggested setting is 432 (3 days of 10 minute intervals) CRON Syntax: * * * * * sar d b interval count >> hostname.sar.date.txt - - - - - | | | | | | | | | + day of week ( 1 - 7) ( Monday = 1) ( Sunday = 0 or 7) | | | +- - - - - - - mont h ( 1 - 12) | | +- - - - - - - - - day of mont h ( 1 - 31) | +- - - - - - - - - - - hour ( 0 - 23) +- - - - - - - - - - - - - mi n ( 0 - 59) Sample CRON Entry: 0 0 * * 5-7 sar d b 600 144 >> zeus.sar.04142009.txt This entry schedules sar to start Friday at 00:00 and end Sunday at 00:00. The sar command pulls statistics every 600 seconds (10 minute) for 24 hours creating 144 intervals. The UNIX redirect >> appends the output to the same file.
5.3 Record Layout First Header Field Description Day of Week Day of week truncated to first three characters Month Month truncated to first thre characters (MMM) Day Day (dd) Time Time (hh:mm:ss) ? Unknown Year Year (YYYY) Second Header Field Description OS Constant HP-UX Server Name Name of the server where sar was run OS Version Version of the OS where sar was run ? Unknown Hardware Family Hardware family where sar was run Date Date (MM/DD/YY) Statistics Heading Description Time (HH:MM:SS). Only on first record in set of statistics. device Device name %busy Percentage of time device was busy servicing a request avgque Average number of I/Os queued r+w/s Total I/Os per second blks/s Total blocks/sec in 512 byte blocks avwait Average I/O queue time (ms) avserv Average I/O service time (ms). AKA response time. Buffer statistics bread/s Total physical reads per second from disk lread/s Total reads per second from buffer cache %rcache Buffer cache hit ratio for reads bwrit/s Total physical writes per second from disk lwrit/s Total writes per second from buffer cache %wcache Buffer cache hit ratio for writes pread/s Total physical reads per second from character device using raw I/O EMC Modeling Tools Data Collection Guide EMC GS/TIS P20/68 Technology Implementation Services
pwrit/s Total physical writes per second from character device using raw I/O 5.4 Sample Output
HP- UX pshqu1 B. 11. 00 U 9000/ 800 02/ 15/ 06
07: 51: 01 devi ce %busy avque r +w/ s bl ks/ s avwai t avser v br ead/ s l r ead/ s %r cache bwr i t / s l wr i t / s %wcache pr ead/ s pwr i t / s 08: 01: 01 c1t 6d0 16. 33 0. 52 21 167 4. 79 10. 12 c2t 6d0 9. 56 0. 52 16 146 4. 87 7. 56 c8t 0d1 0. 05 0. 50 0 1 4. 37 3. 87 c8t 0d2 7. 05 0. 50 16 66 3. 47 6. 59 0 5927 100 14 63 77 169 21 08: 11: 01 c1t 6d0 12. 64 0. 50 16 134 4. 70 10. 29 c2t 6d0 7. 90 0. 50 13 115 4. 82 7. 78 c8t 0d1 0. 03 0. 50 0 0 4. 08 4. 02 c8t 0d2 3. 99 0. 50 13 59 3. 11 4. 16 EMC Modeling Tools Data Collection Guide EMC GS/TIS P21/68 Technology Implementation Services
6 ISERIES (SYSTEM I, IBM I) WRKDSKSTS 6.1 Data Collection Considerations Overflow fields indicated by +++++ must manually be converted to 999.9 in order to successfully import. There is no way to specify the actual value so 999.9 will be lower than the actual value, but it is the largest value you can specify. 6.2 Data Collection Procedure Follow these steps to collect WRKDSKSTS data: 1. Identify the servers and the specific disks that will be replicated. 2. FTP the source code to the iSeries (System i, IBM i) that would be the SRDF source host. 3. Compile the CL (Control Language) program. 4. Add the library containing the compiled program to the library list (if not already there) 5. Run the program. 6. If the program is run multiple times, rename or copy the resulting file (QGPL/DSKSFOREMC) before each time it is run. 7. Collect the QGPL/DSKSFOREMC output file(s). 6.3 Command Syntax The command syntax should be similar to the following and be issued on each host: SBMJOB CMD(CALL PGM(WRKDSKSTS) PARM('Count' 'Interval')) Options Description count Collects statistics for specified intervals Suggested setting is 432 (3 days of 10 minute intervals) interval Collects data every specified minutes Suggested setting is 10 WRKDSKSTS program to write output to spool and then to physical file WRKDSKSTS: PGM PARM( &PARM1 &PARM2) / ******************************************************************************/ / * Aut hor : Russ Lewi s, EMC Cor por at i on */ / * Dat e: 3/ 9/ 01 */ / * Desc: Thi s pr ogr ami s pr ovi ded as an exampl e of wr i t i ng out put f r omt he */ / * WRKDSKSTS command t o a spool f i l e. The spool f i l e( s) out put wi l l */ / * t hen be copi ed & combi ned i nt o physi cal f i l e QGPL/ DSKSFOREMC. */ / * The pur pose i s t o al l ow capt ur i ng of KB per I / O, separ at ed by */ / * r eads oper at i ons ver sus wr i t e oper at i ons, whi ch i s avai l abl e */ / * f r omt he WRKDSKSTS cmd i n OS/ 400. The st at i st i cs f or WRKDSKSTS */ / * ar e r eset bet ween each i t er at i on. 2 char act er f or mat par amet er s */ / * must be passed t o t hi s pr ogr am, 4 byt es f or t he number of */ / * i t er at i ons t o r un, and 4 byt es f or t he number of mi nut es t o wai t */ / * bet ween each i t er at i on. I f bl anks or zer oes ar e passed as */ / * par amet er 1, t he def aul t i s t o r un f or 24 i t er at i ons. I f bl anks */ / * or zer oes ar e passed f or par amet er 2, t he del ay t i me bet ween each */ / * i t er at i on wi l l be 60 mi nut es. */ / * */ / * Exampl e - SBMJ OB CMD( CALL PGM( WRKDSKSTS) PARM( ' 0012' ' 0005' ) ) */ / * wi l l r un 12 i t er at i ons of t he WRKDSKSTS command wi t h a 5 mi nut e */ / * del ay bet ween each i nt er val . */ / * */ EMC Modeling Tools Data Collection Guide EMC GS/TIS P22/68 Technology Implementation Services
/ * Thi s pr ogr ami s not par t of any EMC pr oduct & may be used and/ or */ / * modi f i ed as desi r ed. */ / * */ / * Modi f i cat i on Hi st or y: */ / * */ / * 11/ 01/ 04 RJ L - Changed &PARM1, &I TERATI ONS, &COUNT and &SPOOLCOUNT f r om */ / * 2 t o 4 or 5 byt e var i abl es t o al l ow > 99 i t er at i ons t o r un. */ / * Al so changed f i l e si ze of QGPL/ DSKSFORMEMC t o *NOMAX t o */ / * al l ow f or mor e i t er at i ons wi t h l ar ge conf i gur at i ons. */ / ******************************************************************************/ DCL VAR( &PARM1) TYPE( *CHAR) LEN( 4) DCL VAR( &PARM2) TYPE( *CHAR) LEN( 4) DCL VAR( &I TERATI ONS) TYPE( *DEC) LEN( 4 0) VALUE( 24) DCL VAR( &I TVMI NUTES) TYPE( *DEC) LEN( 4 0) VALUE( 60) DCL VAR( &DELAYSECS) TYPE( *DEC) LEN( 6 0) VALUE( 0) DCL VAR( &COUNT) TYPE( *DEC) LEN( 4 0) VALUE( 0) DCL VAR( &SPOOLCOUNT) TYPE( *DEC) LEN( 5 0) VALUE( 0)
I F COND( &PARM1 *GT ' 0000' ) + THEN( CHGVAR VAR( &I TERATI ONS) VALUE( &PARM1) ) I F COND( &PARM2 *GT ' 0000' ) + THEN( CHGVAR VAR( &I TVMI NUTES) VALUE( &PARM2) )
WRI TEFI LE: DLTF FI LE( QGPL/ DSKSFOREMC) MONMSG MSGI D( CPF2105) CRTPF FI LE( QGPL/ DSKSFOREMC) RCDLEN( 132) + TEXT( ' Tempor ar y Physi cal Fi l e f or + WRKDSKSTS out put ' ) SI ZE( *NOMAX)
/ **** Add 2 t o count er each t i me t hr ough t o ski p over t he "ext r a" spool ****/ / **** f i l e cr eat ed by t he st at i st i cs r eset dur i ng each pass. ****/ CHGVAR VAR( &COUNT) VALUE( &COUNT + 2) GOTO CMDLBL( LOOP2)
ENDPGM: ENDPGM 6.4 Record Layout First Header Field Description Licensed program name for the OS OS version EMC Modeling Tools Data Collection Guide EMC GS/TIS P23/68 Technology Implementation Services
Unknown Title Constant Work with Disk Status Date Date (MM/DD/YY) Time Time (HH:MM:SS) Statistic pre-header Field Description Constant Elapsed time Interval Elapsed time (HH:MM:SS) Constant System name Server Name Server name Statistics Heading Description Unit System assigned number identifying a specific disk Type The disk type Size (M) Disk space (MB) % Used Percentage of disk space allocated I/O Rqs Total I/Os Request Size (K) Average I/O size (KB) Read Rqs Total read IOPS Write Rqs Total write IOPS Read (K) Average Read I/O size (KB) Write (K) Average Write I/O size (KB) % Busy Percentage of time device was busy servicing a request ASP Auxiliary Storage Pool ID Protection Type
6.5 Sample Output 5722SS1 V5R2M0 020719 Wor k wi t h Di sk St at us 10/ 13/ 04 11: 07: 28 Page 1 El apsed t i me . . . . . . . : 01: 00: 00 Syst emname . . . . . . . : A005 Si ze % I / O Request Read Wr i t e Read Wr i t e % - - Pr ot ect i on- - Uni t Type ( M) Used Rqs Si ze ( K) Rqs Rqs ( K) ( K) Busy ASP Type St at us Compr essi on 1 6717 6442 68. 9 22. 6 6. 9 3. 1 19. 4 7. 6 6. 7 3 1 DPY ACTI VE 2 4326 30769 88. 8 33. 6 8. 1 3. 0 30. 6 9. 6 8. 0 1 1 DPY ACTI VE 3 4326 30769 91. 9 40. 9 7. 0 2. 1 38. 7 7. 9 7. 0 1 1 DPY ACTI VE 4 4326 30769 92. 4 31. 1 8. 1 2. 0 29. 0 10. 0 7. 9 1 1 DPY ACTI VE 5 4326 30769 92. 2 35. 6 7. 9 2. 6 32. 9 9. 1 7. 8 2 1 DPY ACTI VE 34 6717 6442 69. 0 41. 3 8. 3 4. 5 36. 8 8. 5 8. 2 5 1 DPY ACTI VE 35 6717 6442 68. 9 38. 1 8. 0 4. 2 33. 8 7. 3 8. 1 5 1 DPY ACTI VE 36 6717 6442 68. 9 41. 4 6. 8 4. 4 36. 9 7. 5 6. 8 5 1 DPY ACTI VE 38 4326 30769 92. 4 32. 1 8. 1 2. 3 29. 8 9. 9 7. 9 2 1 DPY ACTI VE EMC Modeling Tools Data Collection Guide EMC GS/TIS P25/68 Technology Implementation Services
7 LINUX IOSTAT 7.1 Data Collection Considerations Linux iostat formats may vary from version to version with some versions providing additional fields while others do not. There are generally three types of UNIX OS device driver name types internal type drives (the cciss/* types), the sd* types, and the emcpower* types. If PowerPath is installed on the host, then only the emcpower* (PowerPath) device driver names have valid iostat perf data and all of the lower level device driver metrics are not reliable and/or not even valid. (This is because of how the Linux/UNIX kernel works with the PowerPath kernel device driver.) Multiple emcpower devices For the sd* device driver names, then just like with Solaris you will see (as an example, sddj and then sddj1) possibly multiple entries for the same basic host device. In this example, sddj is totally encompassing for all subcomponent partitions (i.e. sddj shows total IOPs for sddj1 and sddj2 and any other partitions). Because of this, we need to strip out all numbers after the primary device driver name (supported with BCSD v1.6.2 parsers and above) as they are overlaps of the primary and if you dont, then the total sum of metrics like IOPs will be incorrect (doubled/tripled/etc.) from what is really being requested. (In other words, only the primary sd entry will be inclusive and if subsequent partition entries are also included in any totals, then the totals will be wrong.) The command can be scheduled in cron or other scheduling products for the desired capture period. 7.2 Data Collection Procedure Follow these steps to collect iostat data: 1. Identify the oservers and the specific LUNS (disks) that will be replicated. 2. Run iostat for a short duration and verify the output matches the required format (see Record Layout and Sample Output below). 3. On each host, run iostat from the command line or optionally create a cron entry. 4. Collect the iostat output file(s). 7.3 Command Syntax The command syntax should be similar to the following and be issued on each host: #iostat d x t interval count >> hostname.iostat.date.txt Options Description -d Displays only the device utilization report -x Displays extended format -t Displays the time of each report interval interval Collects data every specified seconds Suggested setting is 600 (10 minutes) count Collects statistics for specified intervals Suggested setting is 432 (3 days of 10 minute intervals) EMC Modeling Tools Data Collection Guide EMC GS/TIS P26/68 Technology Implementation Services
CRON Syntax: * * * * * iostat d x t interval count >> hostname.iostat.date.txt - - - - - | | | | | | | | | + day of week ( 1 - 7) ( Monday = 1) ( Sunday = 0 or 7) | | | +- - - - - - - mont h ( 1 - 12) | | +- - - - - - - - - day of mont h ( 1 - 31) | +- - - - - - - - - - - hour ( 0 - 23) +- - - - - - - - - - - - - mi n ( 0 - 59) Sample CRON Entry: 0 0 * * 5-7 iostat d x t 600 144 >> zeus.iostat.04142009.txt This entry schedules iostat to start Friday at 00:00 and end Sunday at 00:00. The iostat command pulls statistics every 600 seconds (10 minute) for 24 hours creating 144 intervals. The UNIX redirect >> appends the output to the same file. 7.4 Record Layout Header1 (may not exist) Field Description Date Date statistics were started Time Time statistics were started Year Year statistics were started Header2 Field Description OS Constant Linux OS Version Version of the OS where iostat was run Server Name Name of the server, in parenthesis, where iostat was run Date Date preceded by a tab (MM/DD/YYYY) CPU Info CPU type and quantity Time Field Description Date Date (MM/DD/YYYY) May not exist Time Time (HH:MM:SS AM/PM) or (HH:MM:SS) Extended Statistics (Old format) Heading Description Device: Device name rrqm/s Total read requests merged per second wrqm/s Total write requests merged per second r/s Total reads per second EMC Modeling Tools Data Collection Guide EMC GS/TIS P27/68 Technology Implementation Services
w/s Total writes per second rsec/s Total sectors read per second wsec/s Total sectors written per second avgrq-sz Average I/O size (KB) avgqu-sz Average queue length await Average I/O queue and service time (ms). AKA response time. svctm Average I/O service time (ms) %util Percentage of CPU time I/O requests were issued to the device Extended Statistics (New format) Heading Description Device: Device name rrqm/s Total read requests merged per second wrqm/s Total write requests merged per second r/s Total reads per second w/s Total writes per second rsec/s Total sectors read per second wsec/s Total sectors written per second rkB/s Total kilobytes read per second wkB/s Total kilobytes written per second avgrq-sz Average I/O size (KB) avgqu-sz Average queue length await Average I/O queue and service time (ms). AKA response time. svctm Average I/O service time (ms) %util Percentage of CPU time I/O requests were issued to the device
8 MAINFRAME (SMF/CMF) 8.1 Data Collection Considerations Mainframes write SMF and/or CMF data. This data contains binary information and much more data than needed for modeling. A program to extract and analyze this data is available and called the Mainframe Assembler Kit. This program is written in assembler and is dependant on specific SMF record types being recorded on each LPAR. The use of Mainframe SMF requires the following SMF records: 70 subtype 1, 73 74 subtype 1 and 5 78 subtype 3 The shortest SMF interval recording time the assembler program will accept is five minutes. In a multi-frame, multi-LPAR environment, it may not be possible for all systems to complete the writing of SMF interval data within the confines of a smaller recording interval, so five minutes is the smallest practical interval allowed by the program. The assembler program, and the EMC remote replication modeling tools, do not support mixed SMF recording intervals. If there are systems that use different SMF recording intervals, the SMF data with like intervals are processed together, creating a mainframe CSV file for each different recording interval. As these CSV files have different collection intervals, the EMC remote replication modeling tools require separate projects for each of these as the data can not be combined into composite views. The assembler program must have SMF (RMF record 74 subtype 5) data for ALL DEVICES. The cache counters contained in the RMF 74 subtype 5 records are maintained in the control unit and are universal to all systems. Users therefore often only record the RMF 74 subtype five records on one LPAR to avoid recording redundant data. The LPAR that records the RMF 74 subtype 5 records must have all disk devices varied online or cache statistics for the offline devices will not be recorded. This will result in incomplete input data and inaccurate results. The CMF records written to the SMF dataset are a superset that includes the CMF-specific records and the RMF records used by the assembler program. Users often however, suppress the recording of the RMF records because they use CMF reports in their place and do not want to record data they are not using. The assembler program does not support the native CMF record formats. If the required RMF record types are not being recorded, the user must temporarily enable RMF recording during the data collection period. EMC Modeling Tools Data Collection Guide EMC GS/TIS Technology Implementation Services
Figure 1. Mainframe SMF and CMF Data Collection Process Process SMF/CMF Onsite or Offsite? If possible, run the Mainframe Assembler kit at the customer site to create the CSV file necessary for the EMC remote replication modeling tools. This is usually the quickest and easiest way to process the data and minimizes the amount of data being transferred across the network. Alternatively, the required SMF/CMF records can be sent to EMC corporate via FTP or 3490 tape, processed on the Hopkinton mainframe, thus creating the CSV file for the EMC remote replication modeling tools. 8.2 Mainframe Assembler Kit The Mainframe Assembler Kit is available in one of several ways: 5. Packaged in the BCSD install zip file as Mainframe Assembler Kit vX.zip. 6. For EMC internal users only, in the EMC GS Tools eRoom in the GS Tools Common Components folder. It is packaged in the file called Mainframe Assembler Kit vX.zip which can be downloaded from http://ctseroom02.corp.emc.com/eRoom/tsfuncprac/TSTools/0_53238. If you do not have access, please email TS_Tools_Support@emc.com to request access. The kit contains all necessary programs, sample JCL and a Users Guide. 8.2.1 Additional notes Inputs Control Unit FICON Channel Paths Data File, user parameters Are there FICON channels? Customer Onsite Approach EMC Offsite Approach Get Mainframe Assembler Kit Install Assembler Kit Onsite Customize and run RMFCUGEN Customize and run ET000CSV Retrieve flat-file from Mainframe (as CSV file) TRSMAIN SMF/ CMF data FTP data to EMC SMFDUMP SMF/ CMF data Dump to Tape and post to EMC Obtain TSOid for Remote MF access Restore FTP or Tape data to E01 Yes No EMC Modeling Tools Data Collection Guide EMC GS/TIS Technology Implementation Services
Sorted RMF (and CMF) record types 70 subtype 1, 73, 74 subtype 1, 74 subtype 5, and 78 subtype 3 (74 subtype 1 and 74 subtype 5 are the only record types needed if all channels are ESCON) The CSV file is sorted by DATE, TIME, and descending WRITES. Volume Inclusion/Exclusion If not replicating all volumes you can include or exclude using SYSIN control cards in ETALLCSV or during data import into the EMC remote replication modeling tools. The difference is: If done in ETALLCSV, the CSV will strip out the volsers you specified and cannot be re-included in the remote replication tool unless ETALLCSV is rerun. The advantage is a smaller output file and quicker processing by the EMC remote replication modeling tools. If done in the remote replication tool, the import include/exclude filters can be used against the same input file multiple times depending on the date to be modeled. The disadvantage is a larger output file and slower processing by the remote replication tool. 8.3 Onsite Processing Follow these steps to collect mainframe data: 1. Identify the LPARs and the specific volumes that will be replicated 2. Install the Mainframe Assembler Kit 3. Modify and run the appropriate JCL 4. Collect the output file(s) Optionally Upload Output to EMC FTP Site Follow these steps to upload the CSV file(s) to EMCs FTP site: 1. Optionally zip the CSV file to reduce transfer times 2. Use FTP via the ISPF Command Shell 3. Connect to FTP.EMC.COM 4. Logon as user anonymous, password should be your email address 5. Remain in ASCII Transfer Mode 6. cd incoming/customer 7. put CSV-OUTPUT-FILE CSV FILENAME ON FTP SITE 8. Confirm that the CSV file was transmitted and QUIT 8.4 Offsite Processing Step 1 Extract the Required SMF/CMF Records The TRSMAIN program does not support tape, move the dataset to disk first!!! Edit and run the below SMFDUMP JCL to dump the Type 70, 73, 74 and 78 records to an output dataset. //SMFDUMP J OB (EMC),,CLASS=S,MSGCLASS=X,MSGLEVEL=(1,1), // NOTIFY=&SYSUID //************************************************** //* This job will collect SMF - Type 70, 73, 74 and 78 (dataset //* level statistics) for I/O performance analysis //************************************************** EMC Modeling Tools Data Collection Guide EMC GS/TIS Technology Implementation Services
//* //SMFDUMP EXEC PGM=IFASMFDP //DUMPIN DD DISP=SHR,DSN=INPUT SMF DATASET // DD DISP=SHR,DSN=EXTRA INPUT SMF DSN IF NEEDED //OUTDD1 DD DISP=(,CATLG),DSN=TEMP OUTPUT DATASET, // UNIT=SYSDA,SPACE=(CYL,(25,5),RLSE), // VOL=(,,,5),DCB=(RECFM=VBS,LRECL=32760) //SYSPRINT DD SYSOUT=* //SYSIN DD * INDD(DUMPIN,OPTIONS(DUMP)) OUTDD(OUTDD1,TYPE(70,73,74,78)) Step 2 Terse the SMFDUMP output Documentation on the TRSMAIN program can be found at: http://techsupport.services.ibm.com/390/trsmain.html Use the following JCL to Terse the output dataset prior to FTP transmission: //TRSMAIN J OB (EMC),,CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1), // NOTIFY=&SYSUID,REGION=4M //************************************ //* TRSMAIN with PACK option * //************************************* //TRSMAIN EXEC PGM=TRSMAIN,PARM=PACK //SYSPRINT DD SYSOUT=A //INFILE DD DISP=SHR,DSN=TEMP OUTPUT DATASET //OUTFILE DD DISP=(NEW,CATLG,DELETE),SPACE=(CYL,(50,10),RLSE), // DSN=TERSED DATASET NAME, // DCB=(RECFM=FB,LRECL=1024,DSORG=PS) Step 3 FTP the Tersed File to EMC Use the following instructions to transmit the tersed file to EMCs FTP site. 1. Use FTP via the ISPF Command Shell. Connect to FTP.EMC.COM 2. Logon as user anonymous, password should be your email address. 3. BIN to change to BINARY Transfer Mode. 4. cd incoming/customer 5. put TERSED DATASET NAME TERSED FILENAME ON FTP SITE 6. Confirm that the tersed file was transmitted and QUIT. EMC Modeling Tools Data Collection Guide EMC GS/TIS Technology Implementation Services
Step 4 Login to the Hopkinton Mainframe See Accessing EMCs E01 LPAR for details. Step 5 Retrieve the Tersed SMFDUMP file Transmit the data from our FTP site to our analytics LPAR (E01). 1. Email the TS_Tools_Support mailbox for a TSO userid and password. 2. Logon to the E01 LPAR (IP ADDR=10.243.142.41) 3. Ensure that the target dataset name that you are going to use to receive the tersed file from the FTP site has been pre-allocated with the following DCB: a. LRECL =1024 b. DSORG =PS c. RECFM =FB d. The dataset name should match TERSED INPUT FILE as per below. 4. Use FTP via the ISPF Command Shell. Connect to FTP.EMC.COM 5. Logon as user anonymous, password should be your email address. 6. BIN to change to BINARY Transfer Mode. 7. cd incoming/customer 8. get TERSED FILENAME ON FTP SITE TERSED INPUT FILE (REPLACE) 9. Confirm that the tersed file was received from the FTP site and QUIT. Step 6 Unterse the SMFDUMP file Use the following JCL to unpack the tersed file: //TRSMAIN J OB (EMC),,CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1), // NOTIFY=&SYSUID,REGION=4M //**************************************** //* TRSMAIN with UNPACK option * //**************************************** //TRSMAIN EXEC PGM=TRSMAIN,PARM=UNPACK //SYSPRINT DD SYSOUT=A //INFILE DD DISP=SHR,DSN=TERSED INPUT FILE //OUTFILE DD DISP=(NEW,CATLG,DELETE),SPACE=(CYL,(50,10),RLSE), // DSN=TARGET UNPACKED FILE ON EMC E01 NOTE: The TARGET UNPACKED FILE should have the same attributes as the original TEMP OUTPUT DATASET. 8.5 Record Layout Field Description SITE Site/Customer Identifier IDATE Interval Start Date (MM/DD/YYYY) ITIME Interval Start Time (HH:MM) ISEC Interval Elapsed Seconds (SSSS) EMC Modeling Tools Data Collection Guide EMC GS/TIS Technology Implementation Services
VOLSER Volser DEVNAME Device Type Name SGNAME SMS Storage Group Name UNIT CCUU SSID SSI OEM Vendor CUSER# Control Unit Serial # IREADS All Device Reads (random and sequential) IWRITES All Device Writes (random and sequential) ISEQREADS Sequential Device Reads ISEQWRTS Sequential Device Writes IWRTBLKL Average Write Block Length (Same as IREADBLKL if ESCON) IREADBLKL Average Read Block Length (Same as IWRTBLKL if ESCON) IDEVB True Device Busy All LPARs (PP.PP) IOPS Device I/O/Sec All LPARs ((NNNN.N)N) IRESP Device Response All LPARs (NNNN.N) IQUE Device Avg Q Time All LPARs (NNNN.N) ISERV Device Svc Time All LPARs (NNNN.N) ICNN Device Connect All LPARs (NNNN.N) IDIS Disconnect Time All LPARs (NNNN.N) IPEND Device Pend Time All LPARs (NNNN.N) CARHR Cache Read Hit % X 10 (PPP.P) CARWR Read Percentage X 10 (PPP.P) CAWHR Cache Write Hit % X 10 (PPP.P) CASRR Cache Seq Read % X 10 (PPP.P) CASWR Cache Seq Write % X 10 (PPP.P) CASRH Cache Seq Read Hit % X 10 (PPP.P) CASWH Cache Seq Write Hit % X 10 (PPP.P) CADSR Cache Destage Count
8.7 Accessing EMCs E01 LPAR Before you attempt to use the z/OS TSO LPAR you need to be familiar with TSO, ISPF and JCL. Additionally, you need to email TS_Tools_Support in order to obtain access to a TSOid and Password for temporary use. Keep in mind that you must be behind EMCs Firewall in order to access this LPAR. The first step in getting access to the LPAR, is obtaining and installing a 3270 emulator. EMC uses TN3270 which can be downloaded from http://mfe01.lss.emc.com. The site will look something like this:
Now click on Get Vista TN3270. This will download the install program to your PC. Once downloaded, install and then start it. EMC Modeling Tools Data Collection Guide EMC GS/TIS P37/68 Technology Implementation Services
This will start a new terminal session:
When prompted, type the host IP name mfe01.lss.emc.com, then click Connect.
Just type in the supplied TSOid and password to bring up ISPF. As there is only a small pool of TSOids, your assigned TSOid could be shared, so please do not acquire a session if it is already in use. EMC Modeling Tools Data Collection Guide EMC GS/TIS P39/68 Technology Implementation Services
9 MIRRORVIEW/A LOR (LOCALITY OF REFERENCE) MirrorView/A LOR (locality of reference) provides host-based (Windows only) I/O statistics for modeling MirrorView/A replication solutions with increased accuracy. This tool attempts to mimic what MirrorView/A will actually do if this same workload was run on a CLARiiON array with MirrorView/A active. This is done by determining the locality of reference and how that impacts COFW for snapshots and data movement (SnapView and Incremental SAN Copy). 9.1 Data Collection Considerations There are two steps that need to be run. The first step creates a trace file and the second step uses that trace file as input and creates an analysis file for import. Collection will cease when one of three conditions occurs: when a non-zero duration is complete, when the output file reaches a size of max-output-size, or when the user executes locality with the stop operation. When importing LOR files into the EMC remote replication modeling tools, the Time Zone must be set to GMT. Warning! This tool can produce many GBs of trace data depending on the number of volumes attached to the server and, the sub-period and duration specified. 9.2 Acquiring the MirrorView/A LOR Tool The Mainframe Assembler Kit is available in one of two ways: 1. Packaged with the EMC remote replication modeling tools in the tools install subfolder called MV/A LOR Tool. 2. In the EMC GS Tools eRoom in the GS Tools Common Components folder. It is packaged in the file called MirrorView-A LOR Tool.zip which can be downloaded from: http://ctseroom02.corp.emc.com/eRoom/tsfuncprac/TSTools/0_5499c If you do not have access, please email TS_Tools_Support@emc.com to request access. If you are a non-EMC user, email the support mailbox above and provide the contact details of your EMC account team after which the kit will be made available to you. The kit contains the program and a Users Guide. 9.3 Data Collection Procedure Follow these steps to collect the LOR tool data: 1. Identify the servers and the specific LUNS (disks) that will be replicated 2. On each host, run locality trace from the command line 3. Once the locality trace command has completed, run locality analyze from the command line 4. Collect the LOR output file(s) 9.4 Command Syntax Create Trace File The command syntax should be similar to the following and be issued on each host: locality trace file max-output-size duration Options Description file File name for trace data (should end with the file extension .elt) max-output- Specifies a number [modifier] where modifier may be k or K (kilobytes), or m or M (megabytes), EMC Modeling Tools Data Collection Guide EMC GS/TIS P40/68 Technology Implementation Services
size and must be at least 1 MB. duration The duration is specified as number [modifier] where modifier can be s or S (seconds, default) m or M (minutes), h or H (hours), or d or D (days) Suggested setting is 3d (3 days) Analyze Trace File The analyze must run on same server where .elt file was created. The command syntax should be similar to the following and be issued on each host: locality analyze file period sub-period chunk-size grain-size output-prefix Options Description file File name for trace data (should end with the file extension .elt) period MirrorView/A update period (must match what is specified in the EMC remote replication modeling tools or you may get erroneous results) sub-period Collection interval (1 minute default and must be a divisor of the MirrorView/A update period specified) Suggested setting is 5 minutes to minimize file size and allow period changes in increments of 5 minutes chunk-size Should always be 64K (used to determine reserved LUN pool size and max COFW operations/sec) grain-size FLARE 14/16 should be 16KB and FLARE 19+ should always be 2KB (used to determine network throughput) output-prefix Prefix applied to CSV output files Example A typical use of the locality program: locality trace q:\logs\JetStress.elt 200 24h locality analyze q:\logs\JetStress.elt 1h 5m 64k 2k 1hour The first command begins a trace into q:\logs\JetStress.elt. The trace will run for 24 hours, and will stop collecting data once 24 hours is reached or q:\logs\JetStress.elt reaches 200 MB. The second command will analyze the data for a 1 hour MirrorView/A update cycle (which would result in a 2 hour RPO), reporting incremental CoFW data every 5 minutes, using the default 64KB SnapCopy chunk size and default 2KB MirrorView/A transfer size and will create the files 1hour.csv and 1hour_log.csv. 9.5 Sample Output LoR ver si on, 0. 7 Ti mest amp, 7/ 20/ 2007 16: 26: 05 PM di sk, seconds, per i od, # r eads, # wr i t es, byt es r ead, byt es wr i t t en, chunk si ze, modi f i ed chunks, over wr i t t en chunks, cof w chunks, chunk si ze, modi f i ed chunks, over wr i t t en chunks, cof w chunks 0, 60, 1, 0, 33, 0, 107008, 65536, 5, 28, 0, 2048, 50, 25, 0 0, 120, 0, 0, 9, 0, 32256, 65536, 3, 7, 2, 2048, 18, 6, 12 0, 120, 1, 0, 9, 0, 32256, 65536, 3, 7, 2, 2048, 18, 6, 12 0, 180, 1, 0, 24, 0, 93696, 65536, 5, 20, 2, 2048, 50, 19, 12 0, 240, 0, 0, 34, 0, 125440, 65536, 4, 31, 2, 2048, 43, 50, 12 0, 240, 1, 0, 34, 0, 125440, 65536, 4, 31, 2, 2048, 43, 50, 12 0, 300, 1, 0, 9, 0, 36864, 65536, 2, 8, 1, 2048, 19, 8, 1 0, 360, 0, 0, 28, 0, 100864, 65536, 3, 26, 0, 2048, 32, 43, 0 0, 360, 1, 0, 28, 0, 100864, 65536, 3, 26, 0, 2048, 32, 43, 0 0, 420, 1, 0, 31, 0, 108032, 65536, 4, 27, 2, 2048, 47, 34, 12 0, 480, 0, 0, 16, 0, 56320, 65536, 3, 14, 2, 2048, 20, 22, 12 0, 480, 1, 0, 16, 0, 56320, 65536, 3, 14, 2, 2048, 20, 22, 12 EMC Modeling Tools Data Collection Guide EMC GS/TIS P41/68 Technology Implementation Services
10 NAVISPHERE ANALYZER (NAR) 10.1 Data Collection Considerations for metaLUNs and Thin LUNs 10.1.1 RAW files With raw NAR file support, conversion to CSV is no longer necessary. Note that loading of FLARE 28 and older raw NAR files may fail due to a bug in FLARE which causes an incorrect earliest polltime or Error parsing MetaLUN Stats. Converting these files to CSV and loading the NAR CSV files into BCSD solves this problem. 10.1.2 CSV files For FLARE 28 and above and you can specify metaLUNs and thin LUNs in the NAR to CSV conversion allowing you to create an include or exclude list based on meta-heads, otherwise you must specify all of the meta- members within a metaLUN for the meta to be properly included in the modeling. 10.2 Data Collection Procedure Step 1 Start Analyzer Performance Logging As the data is shared between BOTH SPs, you only need to collect and process a single SPs files, but it may be desired to retrieve NAR files from both SPs in case one file is corrupt. For a MirrorView/A analysis, the collection period should be specified as a divisor of the desired cycle time which is half of the desired RPO time. If you expect an RPO of 2 hours, then the data should be collected in 1 hour intervals or less, such as 30 minutes or even 10 minutes. For any other analysis, the time interval should be specified as 10 minutes or less. Changing the narinterval, which affects both SPs, while logging is active will result in a file with multiple collection intervals. To prevent this, you can first determine if logging is active and what the current collection interval is. If archiving is enabled, then you must stop Analyzer, change the Archive Interval, and then start Analyzer. When you stop analyzer, the current archive data will be saved in a new archive. Use the analyzer STOP command, then the START command to start Analyzer running with the new narinterval. 1. Check if logging is active naviseccli -address <SP_IP_addr> -user <username> -password <pwd> -scope <0|1|2> analyzer -status 2. Display the archive interval naviseccli -address <SP_IP_addr> -user <username> -password <pwd> -scope <0|1|2> analyzer -get 3. If necessary, change the narinterval (remember to enable Periodic Archiving) and restart logging naviseccli -address <SP_IP_addr> -user <username> -password <pwd> -scope <0|1|2> analyzer stop naviseccli -address <SP_IP_addr> -user <username> -password <pwd> -scope <0|1|2> analyzer -set -narinterval <interval in seconds> -periodicarchiving 1 naviseccli -address <SP_IP_addr> -user <username> -password <pwd> -scope <0|1|2> analyzer start Step 2 Copy NAR archive file(s) to your local hard drive List all available NAR files stored on the array naviseccli -address <SP_IP_addr> -user <username> -password <pwd> -scope <0|1|2> analyzer archive -list Retrieve a specific NAR file EMC Modeling Tools Data Collection Guide EMC GS/TIS P42/68 Technology Implementation Services
naviseccli -address <SP_IP_addr> -user <username> -password <pwd> -scope <0|1|2> analyzer -file <filename.nar> -location <pathname> -overwrite <y|n> -retry <times> Retrieve all Files naviseccli -address <SP_IP_addr> -user <username> -password <pwd> -scope <0|1|2> analyzer all -location <pathname> -overwrite <y|n> -retry <times> For raw NAR files, no further processing of the files is required. Step 3 Convert NAR file(s) to CSV file(s) FLARE 30 and Above CLI Host LUNs (allows you to specify meta-heads only) You can use the latest version of Navisphere CLI unless your NAR file is pre-FLARE 26. For include/exclude lists, meta-heads must be specified and not the meta-members (meta-heads and not meta-members are presented to the host) naviseccli analyzer archivedump data <filename(s)> -out <filename(s).csv> - object hl -format on,pt,rb,rs,rio,wb,ws,wio FLARE 28 and Above NAR FLARE LUNs, Meta-Members and Thin LUNs For include/exclude lists, meta-members must be specified and not the meta-heads naviseccli analyzer archivedump data <filename(s)> -out <filename(s).csv> - object l tl -format on,pt,rb,rs,rio,wb,ws,wio FLARE 28 and Above NAR Meta-Heads and Thin LUNs For include/exclude lists, meta-heads must be specified and not the meta-members naviseccli analyzer archivedump data <filename(s)> -out <filename(s).csv> - object ml tl -format on,pt,rb,rs,rio,wb,ws,wio FLARE 24 and 26 NAR For include/exclude lists, meta-members must be specified and not the meta-heads naviseccli analyzer archivedump data <filename(s)> -out <filename(s).csv> - object l -format on,pt,rb,rs,rio,wb,ws,wio 10.2.2 Sample batch file to convert all NAR files in a directory to CSV files Replace the naviseccli command with any of the Step 3 Convert NAR file(s) to CSV file(s) commands. The example is using FLARE 30 and Above CLI Host LUNs (allows you to specify meta-heads only) pat h "C: \ Pr ogr amFi l es\ EMC\ Navi spher e CLI "; %pat h%
f or %%f i n ( *. nar ) do naviseccli analyzer archivedump data <filename(s)> -out <filename(s).csv> -object hl -format on,pt,rb,rs,rio,wb,ws,wio 10.3 Command Syntax Options Description -address Specifies the IP address or network name of the targeted SP on the desired storage system -user Specifies the username on the storage system you want to login to -password Specifies the password on the storage system you want to login to -scope Specifies whether the user account on the storage system you want to log in to is local, global, or lightweight directory access protocol (LDAP). 0 (default) indicates EMC Modeling Tools Data Collection Guide EMC GS/TIS P43/68 Technology Implementation Services
global, 1 indicates local, and 2 indicates LDAP. analyzer Issue an Analyzer command archiveretrieve Creates and Retrieves a single archive file archivedump Dumps Analyzer archive files to a CSV file -all Retrieves all files -data Dumps data from archive files -file Specifies the name of the archive file stored on the array to be retrieved -format Specifies which performance characteristics to output and the order in which they appear. The -format switch requires one or more of the codes listed below (not all are listed): ml MetaLUN l LUN tl Thin LUN -get Returns the current values of the performance logging properties -list Lists all the archive files -location Specifies the location that the retrieved file(s) will be stored -logging Resets performance logging -narinterval Changes the polling interval for archive logging. The allowable range is 60 to 3600, and the default is 120. Suggested setting is 600 (10 minutes). -object Specifies the objects for which to collect performance statistics and the order in which to output them. l indicates LUN. -out Dumps multiple files. Specify multiple output filenames separated by commas. The output filenames list corresponds to the dumped filename list. You can dump multiple files into a single file using the -join switch. -overwrite Specifies whether to overwrite an existing archive file on the client system. If the specified archive file already exists and this switch is not used, the command fails. y = Overwrite existing file. n =Do not overwrite existing file -periodicarchiving When you set -periodicarchiving to 1, performance logging automatically creates archive files at periods of 156 samples. The default is 0, no periodic archiving. -reset Stops performance logging. It deletes all the data that the Analyzer provider has collected. Then it starts performance logging. -retry Specifies the number of times to retry a retrieve before giving up completely. A retry occurs only when the actual move of the archive file from the storage system to the client system fails. Suggested setting is 2. setstats -off Disables statistic logging. EMC Modeling Tools Data Collection Guide EMC GS/TIS P44/68 Technology Implementation Services
setstats -on Enables statistic logging. -start Starts performance logging. -stop Stops performance logging. -status Displays performance logging status Note: Refer to Chapter 2 of the Navisphere Analyzer Command Line Interface (CLI) manual for additional information on how to use the CLI. 10.4 Record Layout Field Description Timestamp Date and time (MM/DD/YYYY HH:MM:SS.sss). Header is Unknown in parenthesis, Time zone name in parenthesis, and GMT offset in parenthesis. Commands/sec Reads/sec Number of read commands issued per second. Writes/sec Number of write commands issued per second. Mbytes Read/sec Megabytes read per second. Mbytes Written/sec Megabytes written per second.
11 SOLARIS IOSTAT 11.1 Data Collection Considerations The command can be scheduled in cron or other scheduling products for the desired capture period. 11.2 Data Collection Procedure Follow these steps to collect iostat data: 1. Identify the servers and the specific LUNS (disks) that will be replicated. 2. Run iostat for a short duration and verify the output matches the required format (see Record Layout and Sample Output below). 3. On each host, run iostat from the command line or optionally create a cron entry. 4. Collect the iostat output file(s). 11.3 Command Syntax The command syntax should be similar to the following and be issued on each host: #iostat r Td x interval count >> hostname.iostat.date.txt Field Description -r Comma separated format -Td Displays the timestamp of each report interval -x Displays extended statistics interval Collects data every specified seconds Suggested setting is 600 (10 minutes) count Collects statistics for specified intervals Suggested setting is 432 (3 days of 10 minute intervals) CRON Syntax: * * * * * iostat r Td x interval count >> hostname.iostat.date.txt - - - - - | | | | | | | | | + day of week ( 1 - 7) ( Monday = 1) ( Sunday = 0 or 7) | | | +- - - - - - - mont h ( 1 - 12) | | +- - - - - - - - - day of mont h ( 1 - 31) | +- - - - - - - - - - - hour ( 0 - 23) +- - - - - - - - - - - - - mi n ( 0 - 59) Sample CRON Entry: 0 0 * * 5-7 iostat r Td x 600 144 >> zeus.iostat.04142009.txt This entry schedules iostat to start Friday at 00:00 and end Sunday at 00:00. The iostat command pulls statistics every 600 seconds (10 minute) for 24 hours creating 144 intervals. The UNIX redirect >> appends the output to the same file. EMC Modeling Tools Data Collection Guide EMC GS/TIS P47/68 Technology Implementation Services
11.4 Record Layout Header Field Description Weekday Day of the week (DDD) Month Month (MMM) Day Day (DD) Time Time (HH:MM:SS) Year Year (YYYY) Statistics (Rate) Heading Description device Device name (may be first or last field) r/s Total reads per second w/s Total writes per second kr/s Average kilobytes read per second kw/s Average kilobytes written per second wait Average number of I/Os queued actv Average number of I/Os being serviced svc_t Average I/O service time (ms). AKA response time. %w Percentage of time waiting for service %b Percentage of time device was busy servicing a request 11.5 Sample Output Thu J un 2 21: 00: 00 2005 ext ended devi ce st at i st i cs r / s, w/ s, kr / s, kw/ s, wai t , act v, svc_t , %w, %b, devi ce 14. 4, 12. 5, 110. 1, 60. 3, 0, 0. 4, 16. 3, 0, 17, sd0 0, 0, 1. 5, 0. 1, 0, 0, 1. 5, 0, 0, sd1 0, 0, 0, 0, 0, 0, 0, 0, 0, sd6 0, 0, 0, 0, 0, 0, 2. 5, 0, 0, ssd4 0, 0, 0, 0, 0, 0, 2. 4, 0, 0, ssd5 0, 0, 0, 0, 0, 0, 1. 9, 0, 0, ssd6 0, 0, 0, 0, 0, 0, 2. 2, 0, 0, ssd7 0, 0, 0, 0, 0, 0, 3, 0, 0, ssd8 0, 0, 0, 0, 0, 0, 3, 0, 0, ssd9 0, 0, 0, 0, 0, 0, 2. 5, 0, 0, ssd10 0, 0, 0, 0, 0, 0, 2. 9, 0, 0, ssd11 0, 0, 0, 0, 0, 0, 3. 1, 0, 0, ssd12 EMC Modeling Tools Data Collection Guide EMC GS/TIS P48/68 Technology Implementation Services
12 VMWARE ESXTOP/RESXTOP 12.1 Data Collection Considerations VMware esxtop/resxtop data is only supported by BCSD. Physical Disk data may be redundant and will require include/exclude lists to ensure unique data. Newer versions of esxtop create disk statistics using several different summary levels and naming conventions. For additional information see VMware Knowledge Base article #1014953 at http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1014 953. Current implementations of esxtop create output files with all of the counters with no way to limit this, and thus large numbers of columns are generated. You must run the Filter esxtop for BCSD tool to remove these extra columns before importing into BCSD. Volumes Example 1 All five statistics appear in more than one bucket for the below volumes, because the data is reported at different summary levels. \svr-esx02.amedisys.com\Physical Disk(vmhba0) \svr-esx02.amedisys.com\Physical Disk(vmhba0:0) \svr-esx02.amedisys.com\Physical Disk(vmhba0:0:0) \svr-esx02.amedisys.com\Physical Disk(vmhba0:0:0:0) \svr-esx02.amedisys.com\Physical Disk(vmhba0:0:0:0:1024) Volumes Example 2 Other forms of volume ids may or may not represent the same volume, but are different ways that esxtop summarizes the statistics. \\esxi-4s.bank.ats\Physical Disk(PH-vmhba2:C0:T0:L0) \\esxi-4s.bank.ats\Physical Disk(PN-naa.600508b1001052395358534c57430400-8) 12.2 Data Collection Procedure Follow these steps to collect esxtop statistics: 1. Identify the servers and the specific LUNS (disks) that will be replicated. 2. Run esxtop for a short duration and verify the output matches the required format (see Record Layout and Sample Output below). 3. On each ESX or ESXi server, run esxtop from the command line or optionally create a cron entry. 4. Collect the esxtop output file(s). 5. Filter the esxtop Data using the Filter esxtop for BCSD tool. This is required to remove the extra columns and double-quotes before importing into BCSD.
12.3 Command Syntax The command syntax should be similar to the following and be issued on each host: # nohup esxtop -b -d interval -n count > hostname.date.csv & Options Description -b Specifies batch mode -c config_filename Config file name and path for this set of collection parameters -d interval Collects data every specified seconds Suggested setting is 600 (10 minutes) -n count Collects statistics for specified intervals Suggested setting is 432 (3 days of 10 minute intervals) nohup Allows the command to survive shell logout when used in coordination with an & (optional) & Runs the command as a background process (optional) 12.4 Record Layout Field Description Timestamp Date and time (MM/DD/YYYY HH:MM:SS.sss). Header is Unknown in parenthesis, Time zone name in parenthesis, and GMT offset in parenthesis. Commands/sec Reads/sec Number of read commands issued per second. Writes/sec Number of write commands issued per second. Mbytes Read/sec Megabytes read per second. Mbytes Written/sec Megabytes written per second.
13 VPLEX STATS 13.1 Data Collection Considerations VPLEX data is only supported by BCSD. VPLEX statistics are at the director level and as such, there are no volume level statistics. 13.2 Data Collection Procedure Step 1 Configure monitoring monitor create n <monitor-name> p <collection-period> d /engines/*/directors/* s director.* Step 2 Start monitoring monitor add-file-sink n < sink-name>m <monitor-name> f <file-name> Step 3 Stop monitoring monitor destroy m <monitor-name> f 13.3 Command Syntax Options Description n monitor-name Name of the monitor. The name is appended to the director on which the monitor is configured. p collection-period Frequency at which this monitor collects statistics. Valid arguments are an integer followed by: ms-milliseconds (period is truncated to the nearest second) s-seconds(Default) min-minutes h-hours 0-Disables automatic polling
The default period is 30 seconds. Suggested setting is 10min. d context-path List of one or more comma-separated director(s) for which to display statistics. s stat One or more statistics to monitor, separated by commas n sink-name Name of the monitor. The name is appended to the director on which the monitor is configured. m monitor-name Performance monitor to which to add a console sink. f file-name File to which to send the sinks data. m monitor-name List of one or more names of the monitor(s) to destroy. f Destroy monitors with enabled sinks and bypass confirmation.
13.4 Record Layout Heading Description Object Name LUN Id, RAID type and LUN masked servers. LUN Id is the number after the open bracket "[" and before the semicolon ";". Time Date and time (YYYY-MM-DD HH:MM:SS). AKA Timestamp. director.be-aborts (counts/s) Number of aborted I/O operations per second through the directors back-end ports. director.be-ops (counts/s) Number of I/O operations per second through the directors back-end ports. director.be-ops-read (counts/s) Number of reads per second through the directors back-end ports. director.be-ops-write (counts/s) Number of writes per second through the directors back-end ports. director.be-read (KB/s) Number of kilobytes per second read by the directors back-end ports. director.be-write (KB/s) Number of kilobytes per second written by the directors back-end ports. director.busy (%) Percentage of CPU usage. director.fe-ops (counts/s) Number of I/O operations per second through the directors front-end ports. director.fe-ops-act (counts) Number of active outstanding I/O operations on the directors front-end ports. director.fe-ops-q (counts) Number of queued outstanding I/O operations on the directors front-end ports. director.fe-ops-read (counts/s) Number of reads per second through the directors front-end ports. director.fe-ops-write (counts/s) Number of writes per second through the directors front-end ports. director.fe-read (KB/s) Number of kilobytes per second read by the directors front-end ports. director.fe-write (KB/s) Number of kilobytes per second written by the directors front-end ports. director.heap-used (%) Percentage of memory usage on the director. director.tcp-recv (KB/s) Number of kilobytes received on the specified directors TCP ports. director.tcp-send (KB/s) Number of kilobytes sent on the specified directors TCP ports.
13.5 Sample Output Ti me, di r ect or . be- abor t s ( count s/ s) , di r ect or . be- ops ( count s/ s) , di r ect or . be- ops- r ead ( count s/ s) , di r ect or . be- ops- wr i t e ( count s/ s) , di r ect or . be- r ead ( KB/ s) , di r ect or . be- wr i t e ( KB/ s) , di r ect or . busy ( %) , di r ect or . f e- ops ( count s/ s) , di r ect or . f e- ops- act ( count s) , di r ect or . f e- ops- q ( count s) , di r ect or . f e- ops- r ead ( count s/ s) , di r ect or . f e- ops- wr i t e ( count s/ s) , di r ect or . f e- r ead ( KB/ s) , di r ect or . f e- wr i t e ( KB/ s) , di r ect or . heap- used ( %) , di r ect or . t cp- r ecv ( KB/ s) , di r ect or . t cp- send ( KB/ s) 2010- 08- 21 16: 08: 14, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 71, no dat a, no dat a 2010- 08- 21 16: 18: 13, 0, 1525, 1057, 467, 17888, 14126, 11, 911, 0, 0, 677, 234, 10633, 7063, 74, no dat a, no dat a 2010- 08- 21 16: 28: 13, 0, 1530, 1061, 469, 17871, 14202, 11, 912, 2, 0, 678, 234, 10648, 7101, 76, no dat a, no dat a 2010- 08- 21 16: 38: 13, 0, 1419, 950, 469, 14030, 14206, 17, 909, 0, 0, 675, 234, 10658, 7103, 77, no dat a, no dat a 2010- 08- 21 16: 48: 13, 0, 1295, 826, 469, 9198, 14243, 13, 904, 0, 0, 670, 234, 10595, 7122, 77, no dat a, no dat a 2010- 08- 21 16: 58: 13, 0, 1304, 834, 469, 9511, 14205, 16, 907, 0, 0, 672, 235, 10643, 7103, 77, no dat a, no dat a 2010- 08- 21 17: 08: 13, 0, 1326, 855, 471, 10386, 14244, 14, 909, 0, 0, 674, 236, 10664, 7122, 77, no dat a, no dat a 2010- 08- 21 17: 18: 13, 0, 1297, 829, 467, 9288, 14149, 15, 904, 7, 0, 670, 234, 10609, 7075, 77, no dat a, no dat a 2010- 08- 21 17: 28: 13, 0, 1279, 812, 467, 8599, 14149, 15, 901, 0, 0, 668, 233, 10591, 7074, 77, no dat a, no dat a 2010- 08- 21 17: 38: 13, 0, 1282, 813, 469, 8641, 14169, 15, 904, 1, 0, 670, 234, 10641, 7085, 77, no dat a, no dat a 2010- 08- 21 17: 48: 13, 0, 1277, 808, 468, 8486, 14220, 15, 901, 6, 0, 667, 234, 10562, 7110, 77, no dat a, no dat a 2010- 08- 21 17: 58: 13, 0, 1272, 803, 469, 7981, 14198, 15, 906, 0, 0, 672, 235, 10641, 7099, 77, no dat a, no dat a 2010- 08- 21 18: 08: 13, 0, 1264, 795, 469, 7656, 14224, 15, 908, 0, 0, 673, 235, 10664, 7112, 77, no dat a, no dat a 2010- 08- 21 18: 18: 13, 0, 1270, 797, 472, 7504, 14302, 15, 913, 10, 0, 677, 236, 10699, 7151, 77, no dat a, no dat a
14 WINDOWS PERFMON 14.1 Data Collection Considerations Please keep in mind that Perfmon files may contain a set of summary data labeled _TOTAL. This summary data will be skipped if non-summary disk observations are found. If _TOTAL is the only observation set, then the summary data will be used. 14.2 Data Collection Procedure Once disk IO statistics active, you must start Performance Monitor to capture disk statistics. Select Start | Run, and then enter perfmon.
Select OK (This starts the Performance Monitor console)
Right-click on User Defined, and selecting New | Data Collector Set.
Enter the name for this data collector set and select Create manually.
Select Next
Select Create data logs | Performance counter.
Select Next EMC Modeling Tools Data Collection Guide EMC GS/TIS P56/68 Technology Implementation Services
Select a sample interval (standard is 10 minutes and cannot be less than 1 minute), and then select Add
Select Next
Select the PhysicalDisk counters (selecting PhysicalDisk selects all associated counters) and the LUNs to be measured.
Select Add
Select LUNs to be measured Select ALL Physical Disk Counters EMC Modeling Tools Data Collection Guide EMC GS/TIS P57/68 Technology Implementation Services
Select OK
Your screen should look like this:
Select Next
Select the directory where you would like the data to be saved. EMC Modeling Tools Data Collection Guide EMC GS/TIS P58/68 Technology Implementation Services
Select Next
Select Save and close.
Select Finish
Right-click the newly defined data collector in the right panel, and then select Properties. EMC Modeling Tools Data Collection Guide EMC GS/TIS P59/68 Technology Implementation Services
The default log format is Binary, so be sure to change to Comma Separated.
Select OK
To schedule when to start and stop the collection, right-click the newly defined data collection, and then select Properties. EMC Modeling Tools Data Collection Guide EMC GS/TIS P60/68 Technology Implementation Services
Select OK 14.3 Conversion from BLG to CSV Should you receive a Perfmon BLG (binary) file, you can convert it to a CSV file with the following command which also selects ALL physical disk counters, thus creating a file in the expected format. r el og input_file. bl g - c " \ Physi cal Di sk( *) \ *" - f CSV - o output_file. csv - y To simply convert a Perfmon BLG file to a CSV file: r el og input_file. bl g - f CSV - o output_file. csv - y 14.4 Record Layout Field Description Timestamp Date and time (MM/DD/YYYY HH:MM:SS.sss). Header is Unknown in parenthesis, Time zone name in parenthesis, and GMT offset in parenthesis. Current Disk Queue Length % Disk Time Avg. Disk Queue Length % Disk Read Time EMC Modeling Tools Data Collection Guide EMC GS/TIS P63/68 Technology Implementation Services
Avg. Disk Read Queue Length % Disk Write Time Avg. Disk Write Queue Length Avg. Disk sec/Transfer Avg. Disk sec/Read Avg. Disk sec/Write Average write response time Disk Transfers/sec Disk Reads/sec Total reads per second Disk Writes/sec Total writes per second Disk Bytes/sec Disk Read Bytes/sec Disk Write Bytes/sec Avg. Disk Bytes/Transfer Avg. Disk Bytes/Read Average read I/O size (bytes) Avg. Disk Bytes/Write Average write I/O size (bytes) % Idle Time Split IO/Sec
14.5 Sample Filtered Output "( PDH- CSV 4. 0) ( East er n Dayl i ght Ti me) ( 240) ", "\ \ S00RES174\ Physi cal Di sk( 0) \ Cur r ent Di sk Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 0) \ % Di sk Ti me", "\ \ S00RES174\ Physi cal Di sk( 0) \ Avg. Di sk Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 0) \ %Di sk Read Ti me", "\ \ S00RES174\ Physi cal Di sk( 0) \ Avg. Di sk Read Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 0) \ %Di sk Wr i t e Ti me", "\ \ S00RES174\ Physi cal Di sk( 0) \ Avg. Di sk Wr i t e Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 0) \ Avg. Di sk sec/ Tr ansf er ", "\ \ S00RES174\ Physi cal Di sk( 0) \ Avg. Di sk sec/ Read", "\ \ S00RES174\ Physi cal Di sk( 0) \ Avg. Di sk sec/ Wr i t e", "\ \ S00RES174\ Physi cal Di sk( 0) \ Di sk Tr ansf er s/ sec", "\ \ S00RES174\ Physi cal Di sk( 0) \ Di sk Reads/ sec", "\ \ S00RES174\ Physi cal Di sk( 0) \ Di sk Wr i t es/ sec", "\ \ S00RES174\ Physi cal Di sk( 0) \ Di sk Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( 0) \ Di sk Read Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( 0) \ Di sk Wr i t e Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( 0) \ Avg. Di sk Byt es/ Tr ansf er ", "\ \ S00RES174\ Physi cal Di sk( 0) \ Avg. Di sk Byt es/ Read", "\ \ S00RES174\ Physi cal Di sk( 0) \ Avg. Di sk Byt es/ Wr i t e", "\ \ S00RES174\ Physi cal Di sk( 1) \ Cur r ent Di sk Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 1) \ %Di sk Ti me", "\ \ S00RES174\ Physi cal Di sk( 1) \ Avg. Di sk Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 1) \ % Di sk Read Ti me", "\ \ S00RES174\ Physi cal Di sk( 1) \ Avg. Di sk Read Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 1) \ %Di sk Wr i t e Ti me", "\ \ S00RES174\ Physi cal Di sk( 1) \ Avg. Di sk Wr i t e Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 1) \ Avg. Di sk sec/ Tr ansf er ", "\ \ S00RES174\ Physi cal Di sk( 1) \ Avg. Di sk sec/ Read", "\ \ S00RES174\ Physi cal Di sk( 1) \ Avg. Di sk sec/ Wr i t e", "\ \ S00RES174\ Physi cal Di sk( 1) \ Di sk Tr ansf er s/ sec", "\ \ S00RES174\ Physi cal Di sk( 1) \ Di sk Reads/ sec", "\ \ S00RES174\ Physi cal Di sk( 1) \ Di sk Wr i t es/ sec", "\ \ S00RES174\ Physi cal Di sk( 1) \ Di sk Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( 1) \ Di sk Read Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( 1) \ Di sk Wr i t e Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( 1) \ Avg. Di sk Byt es/ Tr ansf er ", "\ \ S00RES174\ Physi cal Di sk( 1) \ Avg. Di sk Byt es/ Read", "\ \ S00RES174\ Physi cal Di sk( 1) \ Avg. Di sk Byt es/ Wr i t e", "\ \ S00RES174\ Physi cal Di sk( 2) \ Cur r ent Di sk Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 2) \ %Di sk Ti me", "\ \ S00RES174\ Physi cal Di sk( 2) \ Avg. Di sk Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 2) \ % Di sk Read Ti me", "\ \ S00RES174\ Physi cal Di sk( 2) \ Avg. Di sk Read Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 2) \ %Di sk Wr i t e Ti me", "\ \ S00RES174\ Physi cal Di sk( 2) \ Avg. Di sk Wr i t e Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( 2) \ Avg. Di sk sec/ Tr ansf er ", "\ \ S00RES174\ Physi cal Di sk( 2) \ Avg. Di sk sec/ Read", "\ \ S00RES174\ Physi cal Di sk( 2) \ Avg. Di sk sec/ Wr i t e", "\ \ S00RES174\ Physi cal Di sk( 2) \ Di sk Tr ansf er s/ sec", "\ \ S00RES174\ Physi cal Di sk( 2) \ Di sk Reads/ sec", "\ \ S00RES174\ Physi cal Di sk( 2) \ Di sk Wr i t es/ sec", "\ \ S00RES174\ Physi cal Di sk( 2) \ Di sk Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( 2) \ Di sk Read Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( 2) \ Di sk Wr i t e Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( 2) \ Avg. Di sk Byt es/ Tr ansf er ", "\ \ S00RES174\ Physi cal Di sk( 2) \ Avg. Di sk Byt es/ Read", "\ \ S00RES174\ Physi cal Di sk( 2) \ Avg. Di sk Byt es/ Wr i t e", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Cur r ent Di sk Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ %Di sk Ti me", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Avg. Di sk Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ %Di sk Read Ti me", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Avg. Di sk Read Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ %Di sk Wr i t e Ti me", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Avg. Di sk Wr i t e Queue Lengt h", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Avg. Di sk sec/ Tr ansf er ", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Avg. Di sk sec/ Read", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Avg. Di sk sec/ Wr i t e", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Di sk Tr ansf er s/ sec", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Di sk Reads/ sec", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Di sk Wr i t es/ sec", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Di sk Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Di sk Read Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Di sk Wr i t e Byt es/ sec", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Avg. Di sk Byt es/ Tr ansf er ", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Avg. Di sk Byt es/ Read", "\ \ S00RES174\ Physi cal Di sk( _Tot al ) \ Avg. Di sk Byt es/ Wr i t e" "07/ 25/ 2005 17: 27: 21. 627", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0 ", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0" "07/ 25/ 2005 17: 37: 21. 650", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0 ", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0" EMC Modeling Tools Data Collection Guide EMC GS/TIS P65/68 Technology Implementation Services
APPENDIX A TROUBLESHOOTING AND SUPPORT If you followed the appropriate data collection procedure and the data does not import, send a description of your problem and a copy of your file(s) to TS_Tools_Support@emc.com. If the file(s) are large text file(s), please send a zipped copy of the file(s). If the file(s) are too large to email, upload them to the EMC ftp site and send us the location. EMC Modeling Tools Data Collection Guide EMC GS/TIS P66/68 Technology Implementation Services
APPENDIX B FEATURE ENHANCEMENTS AND FEEDBACK B.1 Feature Enhancements Users can make feature enhancement requests for BCSD in the GS Tools Enhancement Request System GERS. Feature requests will be evaluated and if approved, added to future releases as soon as possible. To enter an enhancement request use the link below. http://gstools.corp.emc.com/gers/index.asp
Select the Product Family from the pull down, BCSD
Select Enhancement Request from Problem Type Pull Down Fill in the required information, anything with a red * asterisk is a required field. Problem/Enhancement Summary (25 words or less) Problem/Enhancement Description
Attach files for reference that will assist in reviewing and possibly developing the enhancement request: Samples (Word, Excel etc) Documents References Screenshots (in Word Doc)
Select Submit The GERS system will automatically send you and email with your Request number and status, anytime the status of the request changes you will be notified. DO NOT REPLY to the email it goes to an unmonitored mailbox. If you want to add any comments or review the request click on the link and it will open the request for you. Enter comments in the Add more info: field shown below. EMC Modeling Tools Data Collection Guide EMC GS/TIS P68/68 Technology Implementation Services
B.2 Feedback Users can offer feedback by emailing TS_Tools_Support@emc.com.