You are on page 1of 66

Symmetrix Performance Workshop

April 2012

EMC Education Services

Copyright
Copyright 1996, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC2, EMC, Data Domain, RSA, EMC Centera, EMC ControlCenter, EMC LifeLine, EMC OnCourse, EMC Proven, EMC Snap, EMC SourceOne, EMC Storage Administrator, Acartus, Access Logix, AdvantEdge, AlphaStor, ApplicationXtender, ArchiveXtender, Atmos, Authentica, Authentic Problems, Automated Resource Manager, AutoStart, AutoSwap, AVALONidm, Avamar, Captiva, Catalog Solution, C-Clip, Celerra, Celerra Replicator, Centera, CenterStage, CentraStar, ClaimPack, ClaimsEditor, CLARiiON, ClientPak, Codebook Correlation Technology, Common Information Model, Configuration Intelligence, Configuresoft, Connectrix, CopyCross, CopyPoint, Dantz, DatabaseXtender, Direct Matrix Architecture, DiskXtender, DiskXtender 2000, Document Sciences, Documentum, elnput, E-Lab, EmailXaminer, EmailXtender, Enginuity, eRoom, Event Explorer, FarPoint, FirstPass, FLARE, FormWare, Geosynchrony, Global File Virtualization, Graphic Visualization, Greenplum, HighRoad, HomeBase, InfoMover, Infoscape, Infra, InputAccel, InputAccel Express, Invista, Ionix, ISIS, Max Retriever, MediaStor, MirrorView, Navisphere, NetWorker, nLayers, OnAlert, OpenScale, PixTools, Powerlink, PowerPath, PowerSnap, QuickScan, Rainfinity, RepliCare, RepliStor, ResourcePak, Retrospect, RSA, the RSA logo, SafeLine, SAN Advisor, SAN Copy, SAN Manager, Smarts, SnapImage, SnapSure, SnapView, SRDF, StorageScope, SupportMate, SymmAPI, SymmEnabler, Symmetrix, Symmetrix DMX, Symmetrix VMAX, TimeFinder, UltraFlex, UltraPoint, UltraScale, Unisphere, VMAX, Vblock, Viewlets, Virtual Matrix, Virtual Matrix Architecture, Virtual Provisioning, VisualSAN, VisualSRM, Voyence, VPLEX, VSAM-Assist, WebXtender, xPression, xPresso, YottaYotta, the EMC logo, and where information lives, are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. Copyright 2012 EMC Corporation. All rights reserved. Published in the USA.

Revision Date: March 28, 2012 Revision Number: MR-1CP-SPERET 1.5

EMC Education Service

Document Revision History


Rev # File Name Date

1.4 1.5

SPW_lab_guide.docx SPW_lab_guide.docx

March 2012 April 2012

EMC Education Service

Table of Contents
COPYRIGHT............................................................................................................. 2 DOCUMENT REVISION HISTORY ............................................................................. 3 LAB EXERCISE 1: TOOLS .......................................................................................... 5 LAB 1: LAB 1: LAB 1: LAB 1: PART 1 USING SYMMETRIX PERFORMANCE ANALYZER.............................................6 PART 2 LAUNCHING PERFORMANCE MANAGER....................................................14 PART 3 MAPPING LOCAL ARCHIVES ...................................................................16 PART 4 PERFORMANCE MANAGER FEATURES ......................................................18

LAB EXERCISE 2: ANALYSIS.................................................................................... 24 LAB 2: PART 1 WORKLOAD CHARACTERIZATION ............................................................25 LAB EXERCISE 3: SYMMETRIX FRONT-END ............................................................ 28 LAB 3: LAB 3: LAB 3: LAB 3: PART 1 EFFECT OF I/O SIZE ..............................................................................29 PART 2 GUIDED ANALYSIS ................................................................................31 PART 3 AUTO DESIGN FRONT-END PERFORMANCE ...............................................34 PART 4 24/7 FRONT-END PERFORMANCE ..........................................................36

LAB EXERCISE 4: SYMMETRIX CACHE .................................................................... 38 LAB 4: LAB 4: LAB 4: LAB 4: LAB 4: PART 1 EFFECTS OF RANDOMNESS ....................................................................39 PART 2 GUIDED ANALYSIS................................................................................41 PART 3 ONE HOST REPORTING PROBLEMS ...........................................................44 PART 4 AUTO DESIGN CACHE PERFORMANCE .......................................................46 PART 5 24/7 CACHE PERFORMANCE .................................................................48

LAB EXERCISE 5: SYMMETRIX BACK-END .............................................................. 50 LAB 5: LAB 5: LAB 5: LAB 5: PART 1 GUIDED ANALYSIS ................................................................................51 PART 2 AUTO DESIGN BACK-END PERFORMANCE ..................................................54 PART 3 PREFETCH ANALYSIS .............................................................................56 PART 4 24/7 BACK-END PERFORMANCE .............................................................58

LAB EXERCISE 6: BUSINESS CONTINUITY ............................................................... 60 LAB 6: PART 1 GUIDED ANALYSIS................................................................................61 LAB 6: PART 2 UNEXPECTED NIGHT DELAYS..................................................................63

EMC Education Service

Lab Exercise 1: Tools


Purpose:

To become familiar with the use of Symmetrix Performance Analyzer and Performance Manager

Tasks:

In this lab, you will perform the following tasks:

Use Symmetrix Performance Manager to examine workload characteristics Map and open archives in Performance Manager Use Views and Metrics to explore the archive Create new views, change the appearance of graphs, and export data

EMC Education Service

Lab 1: Part 1 Using Symmetrix Performance Analyzer


Step 1 Action Launch Symmetrix Performance Analyzer by double-clicking the desktop icon URL, or by entering this URL in Internet Explorer: https://hostname:8443/spa If it warns you about security or certificate issues, just confirm in a way that continues the program launch. Log in using the user and password your instructor has provided, or with the default: Login: smc Password: smc It may take up to a minute after the initial launch for the program to become active. Be patient.

EMC Education Service

Step 2 Explore the default Dashboard.

Action

Note that the default view when you log in to SPA is the default Dashboard Heat Map. Examine the Dashboard and identify the components that are actively performing work. a. Record any components that are color coded red indicating 100% utilization. FA/RA Directors: __________________________________ Cache (Cache Partitions if any): ______________________ DA Directors: ______________________________________ Disks: ____________________________________________

b. Click on the components identified above and record their utilization. FA/RA Directors: __________________________________ Cache (Cache Partitions if any): ______________________ DA Directors: ______________________________________ Disks: ____________________________________________

EMC Education Service

Step 3 Overall Symmetrix Performance.

Action

Select the Snapshot view and click on the local Symmetrix in the left.

Use the Utilization Distribution to find out what the overall condition of the Symmetrix components is. Do any of the components have greater than 25% utilization? If so, which components (Hint: click on the bar in the snapshot to show more details)? FA Dirs: ______________________________ Cache: ______________________________ BE Dirs: _____________________________ Disks (Disk Groups): ___________________

Use the Throughput Distribution to determine the Front end MB/s. FA: _________________________MB/s

Are there any Device and/or Storage groups listed in the Snapshot view. Use the Device Groups I/Os Per Second & Response Time Distribution to find the device group (do not include ungrouped devices) having the highest utilization. What rating is the groups response time? Group:_______________________________ Response time rating: ________________

What is the overall IOs Per Second to this Symmetrix? ___________________

EMC Education Service

Step 4 Symmetrix Storage Group Snapshots(1).

Action

Use the navigation panel on the left to locate and click on the Storage Group that you identified in the previous part. Make sure you are still viewing Snapshots.

Use the Storage Group Profile to find IO profile for this Group: Determine if the workload is predominantly reads or writes (%): ___________ Is the workload predominantly random or sequential: ____________________ The overall Hit rate and Miss rate: __________________________ What are the average read and write IO sizes: ______________________ Does this Storage Group contain Thinly Provisioned devices: ______________

EMC Education Service

Step 5 Symmetrix Storage Group Snapshots(2).

Action

Use the navigation panel on the left to locate and click on the Storage Group that is Virtually Provisioned. Make sure you are still viewing Snapshots. Use the Storage Group Profile to find IO profile for this Group: Determine if the workload is predominantly reads or writes (%): ___________ Is the workload predominantly random or sequential: ____________________ The overall Hit rate and Miss rate: __________________________ What are the average read and write IO sizes: ______________________ Record the information relevant to VP for this Storage Group: IO Density by Tier: _____________________________ Symmetrix Tier Capacity Allocation %: _________________________ BE IOs by Tier: ____________________________ BE Throughput by Tier: _______________________ IOs Per Second to this Storage Group:_________________________

Symmetrix Front-End Directors Snapshots(1). Open the sub-folders under the Symmetrix array in the navigation pane on the left and click on the FE Directors folder. Make sure you are still viewing Snapshots. Identify the Director(s) that are performing the most work (IOs Per Second and Utilization. _________________________________ Identify the Port(s) that are performing the most work (MBs Per Second)._____________

EMC Education Service

10

Step 7

Action Symmetrix Front-End Directors Snapshots(2). Open the FE Director folder and choose just one front-end director identified in step(6). Make sure you are viewing Snapshots. What is the IO contribution (%) of this director to the overall workload on the Symmetrix: ______________________ Is this director servicing more reads or more writes: ________________________ What is the Throughput contribution (% and MB) of this director to the overall workload on the Symmetrix: _____________________________ Are both the Front-end ports on this director equally utilized: ___________________

Symmetrix Back-End Directors Snapshots. Click on the BE Directors folder, and make sure you are viewing Snapshots.

Use the Snapshot views to monitor the balance of the back-end directors. Are some directors being utilized more than others? __________ Can you detect similar patterns of usage across two or more directors? _______________________

If you detected imbalance on the front-end directors, but mostly balance on the backend directors, what does this say about the arrangement of data in the array?

EMC Education Service

11

Step 9 Symmetrix Disks Snapshots.

Action

Click the Disks folder, and make sure you are viewing Snapshots.

Use the Snapshot views to determine if any of the disk groups have an average utilization of more than 25%: _________________________________________________________________________ Do any of the disk groups have a peak utilization of more than 25%? ______________

Drill down under the Disks folder to click on a single disk group. Are the disks in the group being utilized fairly evenly? __________________________________________

Compare a few of the disks in your group to see if they have similar traffic patterns.

10

Diagnostic view. Select Diagnostic view. Observe the overall dashboard that is presented for the Symmetrix. Double-click on the Symmetrix and all the available Diagnostic view tabs are presented. Explore the different components. Select the FE Director tab. Then select the FE identified as busy in the earlier part of this exercise. Switch to the Explore tab. Select both the Average Read Response Time and Average Write Response Time and plot the graph. Is there any abrupt change in the response times: ________________________ Diagnostic view can help in quickly identifying changes that have occurred recently. Explore the other components for which Diagnostic view is available.

EMC Education Service

12

Step 11 Real Time view (1).

Action

Select Real Time view. The overall metrics for the Symmetrix in the past hour with finer granularity is displayed. Select the Symmetrix. Then select FE Reqs/sec and CTRL select BE Reqs/sec. Plot the graph. Which of the two is consistently higher: ______________________________

We will discuss the reason for these later in the course. 12 Real Time view (2). Double-click on the Symmetrix. This will provide three tables for FE, BE and RDF Directors. Take some time to explore the metrics available for these components in the Real Time view.

End of Lab 1 Part 1

EMC Education Service

EMC Education Service

13

Lab 1: Part 2 Launching Performance Manager


Performance Archive Used In This Part Symmetrix/000194900180/interval/20091124.btp. All performance archives are available on the desktop of your classroom workstation. They can also be found on the student CD.

Step 1

Action Launch Performance Manager by double-clicking the desktop icon, or by using the EMC > ControlCenter > Performance Manager menu of the Windows Start menu.

Click the Work Offline box in the login dialog and then OK to launch the program.

If a Data Providers dialog appears, just click Cancel.

At the Data Selection dialog, choose the Specific File radio button and browse to locate and open the archive used in this part (listed at the start of the exercise).

EMC Education Service

14

Step 2 Viewing Vital Signs.

Action

Click on the Vital Signs icon, and answer these questions using the graphs. You might use the Window > Cascade or Window > Tile menus to view the Vital Signs better.

What was the maximum I/O rate to the Symmetrix during the collection period? _________ What was the maximum % Hit during the collection period? ________ Does the % Write tend to correspond with the % Hit measure? ________

Use the Window > Close All menu to close all the views.

Close Performance Manager in preparation for the next Part.

End of Lab 1 Part 2

EMC Education Service

15

Lab 1: Part 3 Mapping Local Archives

Performance Archive Used In This Part Symmetrix/000194900180/interval/20091124.btp. All performance archives are available on your classroom workstation, or the student CD.

Step 1

Action Launch Performance Manager using the desktop icon or the Windows Start menu. At the Data Providers dialog, click the Add Group button.

In the Data Provider dialog, Choose Symmetrix from the Class drop down menu, or type in the word if it is not an available choice.

Browse to the Symmetrix folder of your archive setit should be on the Desktop. Click directly on the top-level folder named Symmetrix, not the folder for a particular Symmetrix ID.

Click OK to save the selection and exit the Data Provider dialog.

If you did this correctly, a list of Symmetrix archives should appear in the Data Providers dialog. If not, use the dialog to delete any mistakes and try again.

Click OK to exit the Data Providers dialog.

EMC Education Service

16

Step 3

Action Use the Data Provider and Archives parts of the Data Selection dialognot the Specific File partto launch the archive used in this part. You can now retrieve any archives used in the class using the Data Provider and Archives part of the Data Selection dialog.

You may leave this archive open for the next Part.

End of Lab 1 Part 3

EMC Education Service

17

Lab 1: Part 4 Performance Manager Features


Performance Archive Used In This Part Symmetrix/000194900180/interval/20091124.btp

Step 1

Action Launch Performance Manager and open the archive used with this part, if you have not already done so.

Plot the All Fibre Directors IO Rates graph. This graph (View) can be found in the Dir-Fibre Folder. By default this graph will be plotted as a Line graph.

Graph WizardRibbon Graph. (a) Select the graph you opened (make sure its title bar is not gray) and click the Graph Wizard icon ( ).

(b) Change the Chart Type to Ribbon, and click OK. (c) Are all the directors performing equal amount of IO?______________ (d) Can you detect grouping of directors performing equal amounts of IO?_______ (e) What are the groups? _______________

EMC Education Service

18

Step 4 Graph WizardFill Style.

Action

(a) Click the Graph Wizard icon again. (b) Select the Fill Style tab. (c) Change the Fill Style for the Legend, Display, and Title, to None. (d) Click on Apply. (e) Note the change in appearance of the graph. (f) Close the graph and then launch it again. Were the changes you made retained? __________ (g) Close the graph.

Explore other features of the Graph Wizard throughout the rest of the lab exercises.

EMC Education Service

19

Step 5 Creating a New Data View.

Action

(a) Right-click on the System folder and select New Data View. (b) In the Data View Definition dialog, leave the Class as Symmetrix and change the Identifier to an asterisk (*). What effect does changing the Identifier to an asterisk have? ____________________________________________________________ (c) Select Dir-DA as the Category, ios per sec for the Metric, for All objects. Check the Sum Across Selected Objects and click the Add to Contents button. (d) Repeat (c) for Category Dir-Fibre. (e) Choose Ribbon as the Graph Style. (f) Enter a Name (this will appear under the Systems folder when the definition is created), and a Title. If you are connected to a Repository, check the Public Views box and click OK. What functionality does checking the Public Views box provide? _____________________________________________________________ ___ (g) Plot the graph from the Systems folder. (h) The Ribbon format does not add any utility when the two lines are so dissimilar. Right-click on the View, choose Modify Data View, change the Graph Style to Line, and click OK. (i) Plot the graph again to verify the change.

EMC Education Service

20

Step 6 Metrics Tab.

Action

(a) Click on the Dir-Fibre folder in the top left panel. Select any director. In the metrics panel, click on ios per sec and CTRL-click on requests per sec. (b) Click on the Graph per Object button ( ). The resulting graph now displays both ios per sec and requests per sec. (c) Click on the Insert the graph into Views button ( ).

(d) In the Save Graph as Data View dialog, check both the Generalize Class IDs and Generalize Objects boxes, and give it a name. (e) Click OK to save the new view. (f) What category in the Views tab did the new view appear under? Why?

EMC Education Service

21

Step 7 Single-point Analysis Mode.

Action

(a) Launch the System View Total Throughput to-from hosts view (or any similar line graph view) from the Views tab. (b) Right-click on any part of the graph and enable Single Point Analysis mode. (c) Select the Metrics tab and choose the Dir-Port folder in the top panel. (d) Shift-click to choose all of the director ports that show a non-zero I/Os per second value in the middle panel. (e) Click on the ios per sec measure in the bottom panel. (f) Click the Graph histogram from sorted metric button ( ). The resulting histogram shows the I/O per second of each port at a given time in the time graph. (g) Click the time graph at the point of peak throughput. The histogram will show the status of those objects at that time.

What is the time: __________ and approximate peak throughput? ______________ Record the top two ports and their I/O per sec at this time: ____________________________________________________________________ What is the time (or a time) of the lowest throughput? ____________________ Are the ports receiving any I/Os at that time? ___________________

Experiment with this feature by adding another histogram from the Metrics tab and changing the time. When you are done experimenting, you can close all the graphs you created to end Single Point Analysis mode.

EMC Education Service

22

Step 8

Action Are the values in the graphs sometimes difficult to read? Open any Views or Metrics graph, and then use the Table option. from the Tools menu to show the time and data points that make up the graph. Try this now.

You can copy the data out of a table to spreadsheet or text editor, or you can just save the graph as a CSV file. Select the graph and use File > Save as csv > Graph to save the data. View the output in a spreadsheet program or Notepad.

Try the File > Save Graph as menu to save an image of the graph. Double-click the image to view it.

End of Lab Exercise 1

EMC Education Service

23

Lab Exercise 2: Analysis


Purpose:

To become familiar with the use of Symmetrix Performance Analyzer and Performance Manager

Tasks:

In this lab, you will perform the following tasks:

Analyze the archive to determine average workload characteristics Analyze the archive to determine profile of usage, or the changes in the workload over time Analyze the archive to determine component usage, or the way in which the workload is distributed over the components

EMC Education Service

24

Lab 2: Part 1 Workload Characterization

Performance Archive Used In This Part Symmetrix/000284500356/daily/20021215d.btp

Workload Characteristics
Examine the archive used in this part and record the following information about the Workload Characteristicsthe basic measures for the overall system. Use any graph from the Views or Metrics tab of Performance Manager to get your answers. To get an average number for a measure that is constantly changing, display the measure as a histogram.

Characteristic Average System I/O per second Average System Throughput Average System I/O Size (divide Throughput by I/O per second) Average System Hit Percentage Average System Write Percentage Average System Write Pendings Average Back-End I/O per second (sum I/O per second over all disk directors)

Average Value

EMC Education Service

25

Profiles of Usage Examine the archive to detect usage changes over time. Consider the measures you looked at in the previous part, especially the I/O per second measure. Are there noticeable changes in the characteristics at certain times? Do your best to identify the time periods that differentiate the workload, and record them here. Also note what evidence led you to conclude that a change in the traffic had happened here: increased I/O per second, change in write percentage, etc.

Period 1 2 3 4

Start - End

Evidence

Component Usage Examine the archive to determine what components are being utilized. Answer each of the following questions by examining the I/O per second measure for the indicated components

Step 1 2

Action List any front end ports that are mostly unused: Are any of the front end ports grouped? Do they seem to be sharing the workload for one I/O stream? If so, list the ports:

List any back end directors that are mostly unused:

EMC Education Service

26

Step 4

Action List any physical disks that are mostly unused (if there are many, just write the count out of the total number: 10/30):

Are any of the disks grouped? Do they seem to be sharing the workload for one I/O stream? If so, list the disks (if there are many, just write the count out of the total number: 10/30):

Are any of the devices grouped? Do they seem to be sharing the workload for one I/O stream? If so, list the devices (if there are many, just write the count out of the total number: 10/30):

End of Lab Exercise 2

EMC Education Service

27

Lab Exercise 3: Symmetrix Front-end


Purpose:

To use guided and free-format techniques to detect front-end issues.

Tasks:

In this lab, you will perform the following tasks:

Relate I/O size to performance Use guided steps to analyze front-end issues Analyze front-end issues using techniques you learned in the course

EMC Education Service

28

Lab 3: Part 1 Effect of I/O size

Performance Archive Used In This Part IOSize: Symmetrix/000184501731/analyst/iosize02.btp

This Part uses the IOSize archive. This archive is not real-world data, nor is it intended to simulate any realworld environment. It is simply a test of the effects of I/O characteristics on Fibre Channel and SCSI ports. The activity was generated by single-threaded I/O generation programs (one for each device) that have only one task: trigger a new I/O request immediately after the completion of the previous one. Four distinct traffic cycles were issued, each signified by a change in one of the I/O characteristics.

Examine the archive and fill in the following table regarding the Symmetrix-wide activity. Host Port:______________________________ Metric I/O per second Throughput Cycle 1 Cycle 2 Cycle 3 Cycle 4

Host Port:______________________________ Metric I/O per second Throughput Cycle 1 Cycle 2 Cycle 3 Cycle 4

Since these applications are simply waiting to send a new I/O request after the completion of the previous one, any increase in traffic rate or volume indicates a performance improvement; any decrease, a reduction in performance.

EMC Education Service

29

Questions How do you explain the wide difference in I/O rate across the four cycles?

When was the actual data throughput at its highest: when the I/O rate was at its lowest or at its highest?

The applications used to generate the activity on both ports are the same. Suggest a reason why the performance on both ports is not identical.

End of Lab 3 Part 1

EMC Education Service

30

Lab 3: Part 2 Guided analysis


Performance Archive Used In This Part Email SRDF: Symmetrix/000284701058/daily/20040420d.btp

This archive was recorded at a site that uses Solaris hosts to support an Email application. The application is replicated to disaster recovery site using four SRDF RA1 ports. Another four RA2 ports receive data from the disaster recovery site.

Step 1

Action Look at the Dir-Fibre > All Fibre Directors IO Rates View (in the Views tab).

What director[s] peak at the highest I/O activity? ______________________________ What is the peak I/O rate, and when does it occur? _____________________________ Do the hosts appear to be attached by 2 or more balanced ports? If so, list the port pairings: ______________________________________________________________

EMC Education Service

31

Step 2

Action Look at the Dir-Fibre>Fibre Directors Utilization View (or a similar View).

What is the peak utilization of all Fibre directors? _________________________ When does this peak occur? ____________________________

Since director utilization is calculated from the I/O per second measure, the director causing the highest utilization must be the one with the peak I/O rate that you recorded earlier. From this information, determine how many I/O per second the director would have to be processing to reach 100% utilization: ___________________________

Look at the Dir-Port>Host Ports % Utilization View.

What port[s] peak at the highest port utilization? _______________________________ What is the peak utilization, and when does it occur? ___________________________

EMC Education Service

32

Step 4

Action Turn to the Metrics tab and plot the throughput for all of the Fibre ports.

What is the peak throughput of all ports, and when does it occur? _________________ How much throughput would the port have to be processing to reach 100% utilization? _____________________________________________________________________

How can the port with the peak utilization not be attached to the director with the peak utilization? ____________________________________________________________

Plot the measure that supports your answer. What did you plot? ______________________________________________________________________

End of Lab 3 Part 2

EMC Education Service

33

Lab 3: Part 3 Auto Design Front-end Performance

Performance Archive Used In This Part Automobile Design: Symmetrix/000284500972/revolving/200204181502.btp

This archive was taken from an array using 72GB RAID-1 drives. To help visualize the effects of adding a new database application, a load operation (write, read back and verify) was performed while this archive was created to capture the performance. Your analysis will explain the effects to the IT group and help them plan for future applications to be added to the array

Analyze the front-end performance of the Automobile Design archive. Thoroughly examine any issue (good or bad performance) related to the topics discussed in this module. Put off detailed analysis of the other components until later in the classyou will revisit this archive again once you finish the other modules. Be prepared to back up any claims with appropriate evidence.

You might want to start with a complete characterization as outlined during the Characterization exercise you did previously. This will give you some idea of the basic environment of this archive, including the general workload characteristics and component usage.

Following this, observe the Roadmap measures outlined in the module to detect issues and problems. Finally, look for any minor issues discussed in the module (off-roadmap topics).

Here are some questions you should be able to answer once you have done your analysis:

EMC Education Service

34

Step 1

Action Is the utilization of any of the front end components overly high?

Is the balance across the front end components good?

Is there room for addition of the new applications?

What is the best strategy for adding new applications with regard to the front end hardware?

End of Lab 3 Part 3

EMC Education Service

35

Lab 3: Part 4 24/7 Front-end Performance

Performance Archive Used In This Part 24/7: Symmetrix/000190100720/interval/20080312.btp

This archive shows full-sized DMX array that is used for a variety of applications around the clock. In this growing environment, the IT staff would be very interested in any recommendations for adding new applications to the array.

Analyze the front-end performance of the archive. Thoroughly examine any issue (good or bad performance) related to the topics discussed in this module. Put off detailed analysis of the other components until later in the classyou will revisit this archive again once you finish the other modules. Be prepared to back up any claims with appropriate evidence.

You should be able to answer these questions once you have finished your analysis:

EMC Education Service

36

Step 1

Action Is the utilization of any of the front-end components overly high?

Is the balance across the front end components good?

Is there room for addition of the new applications?

What is the best strategy for adding new applications with regard to the front end hardware?

If there are any issues, what recommendations do you have for resolving them?

End of Lab 3 Part 4

End of Lab Exercise 3

EMC Education Service

37

Lab Exercise 4: Symmetrix Cache


Purpose:

To use guided and free-format techniques to detect cache issues.

Tasks:

In this lab, you will perform the following tasks:

Relate random I/O to performance Use guided steps to analyze cache issues Detect performance problem caused by shared resources Analyze cache issues using techniques you learned in the course

EMC Education Service

38

Lab 4: Part 1 Effects of Randomness

Performance Archive Used In This Part Random: Symmetrix/000184501731/analyst/random01.btp

This Part uses the Random archive. This archive is not real-world data, nor is it intended to simulate any real-world environment. It is simply a test of the effects of I/O characteristics on Fibre Channel and SCSI ports. The activity was generated by single-threaded I/O generation programs (one for each device) that have only one task: trigger a new I/O request immediately after the completion of the previous one. Three distinct traffic cycles were issued, each signified by a change in one of the I/O characteristics.

Examine the archive and fill in the following table regarding the Symmetrix-wide activity.

Metric I/O per second Throughput

Cycle 1

Cycle 2

Cycle 3

Since these applications are simply waiting to send a new I/O request after the completion of the previous one, any increase in traffic rate or volume indicates a performance improvement; any decrease, a reduction in performance.

EMC Education Service

39

Step 1

Action How do you explain the differences in I/O rate across the three cycles? What metrics support your explanation?

In this example, approximately how much of an impact did this characteristic have on the overall I/O rate (give as a percentage increase between the first and last cycle)?

Is the effect more predominant for reads or writes? Explain why.

End of Lab 4 Part 1

EMC Education Service

40

Lab 4: Part 2 Guided Analysis

Performance Archive Used In This Part Email SRDF: Symmetrix/000284701058/daily/20040420d.btp

This archive was recorded at a site that uses Solaris hosts to support an Email application. The application is replicated to disaster recovery site using four SRDF RA1 ports. Another four RA2 ports receive data from the disaster recovery site.

Step 1

Action Look at the System>System Cache Utilization View.

Estimate the average Hit % and Write % from this View: _______________________ From this view alone, answer this question: Is the write hit percentage equal to 100%?

Look at the System>System Total IO Rate View.

What is the peak total I/O rate, and when does it occur? ________________________ Which is predominant I/O type for this array: reads, sequential reads, or writes?

EMC Education Service

41

Step 3

Action Use the Metrics tab to plot the slot collisions. Generate a graph that shows the total slot collisions across the whole array.

Describe how you generated this graph:

Roughly what percentage of the overall I/Os cause a slot collision? _______________

Plot the Device>System bus Kbytes per sec for all devices, and view the result in an Area graph.

What is the peak internal system throughput, and when does it occur? _____________

Use the Metrics tab to plot the System>system max wp limit and System>number write pending tracks on the same graph.

Is there any evidence of reaching system write pending limits? __________ What is the system write pending limit for this array? _________________

EMC Education Service

42

Step 6

Action Plot the Device>write pending count for all devices on the array. Since this is a pre-DMX-2 array, use the technique discussed in class to determine the base device write pending limit.

Is there any evidence of reaching device write pending limits? _________ If so, which devices seem to be hitting the limits the most? ______________________ What is the base device write pending limit for this array? _______________________

Plot the Dir-Fibre>ios per sec and Dir-Fibre>requests per sec for each of the Fibre directors individuallyif you plot several directors in the same graph, it may be too difficult to match the measures up.

What is the approximate ratio of requests / I/Os for the busiest (highest I/O) pair of directors? _______________________________ Is large I/O size a factor in increasing the number of requests in this case? __________ What graph supports your opinion? _________________________________________

What pair of directors shows the highest (peak) requests to I/O ratio? ______________ Is this high ratio sustained or temporary? _____________________________________ Is large I/O size a factor in the large number of requests for these directors? _________

End of Lab 4 Part 2

EMC Education Service

43

Lab 4: Part 3 One host reporting problems


Performance Archive Used In This Part One Host: Symmetrix/000182503445/revolving/200304081410.btp

This part uses the One Host archive. In this simulated environment, three hosts are connected to a single Symmetrix. Each has a different application profile, and is maintained by a different application administrator. At around 13:30 of this day, sun220 experienced a prolonged slowdown. Average response time increased by 40% on this already heavily utilized server, causing a corresponding reduction in the number of records processed. You will be able to identify this period by observing the drop in I/O rate for sun220.

Analyze the archive and report your findings back to the administration team so that they are aware of the cause of the slowdown and any potential solutions.

Basic Configuration Information sun220 using ports: SA01A-0, SA16A-0

hpd370a using ports: FA05B-0, FA12B-0 f50a using ports: FA05A-0, FA12A-0

EMC Education Service

44

Step 1

Action Describe the event (change in traffic issued from hosts) that caused the performance problem:

Why did this event cause a performance issue?

Why did none of the other hosts connected to the same array report performance issues during this period?

What can be done to avoid the problem in the future?

End of Lab 4 Part 3

EMC Education Service

45

Lab 4: Part 4 Auto design Cache performance


Performance Archive Used In This Part Automobile Design: Symmetrix/000284500972/revolving/200204181502.btp

This archive was taken from an array using 72GB RAID-1 drives. To help visualize the effects of adding a new database application, a load operation (write, read back and verify) was performed while this archive was created to capture the performance. Your analysis will explain the effects to the IT group and help them plan for future applications to be added to the array

Analyze the cache performance of the Automobile Design archive. Thoroughly examine any issue (good or bad performance) related to the topics discussed in this module. Put off detailed analysis of the other components until later in the classyou will revisit this archive again once you finish the other modules. Be prepared to back up any claims with appropriate evidence.

You began analyzing the front-end performance of this archive in the previous Lab Exercise. Revisit your findings. Then refer to the Roadmap measures to begin your analysis and consider the issues discussed in the module.

Here are some questions you should be able to answer once you have done your analysis: Step 1 Action Is the hit percentage unusually low for an open systems environment?

EMC Education Service

46

Step 2

Action Are there any System or Device write pending issues?

Is there any evidence of data misalignment?

If there are any issues, what recommendations do you have for resolving them?

End of Lab 4 Part 4

EMC Education Service

47

Lab 4: Part 5 24/7 Cache performance

Performance Archive Used In This Part 24/7: Symmetrix/000190100720/interval/20080312.btp

This archive shows full-sized DMX array that is used for a variety of applications around the clock. In this growing environment, the IT staff would be very interested in any recommendations for adding new applications to the array.

Analyze the system and cache performance of this archive. Thoroughly examine any issue (good or bad performance) related to the topics discussed in this module. Put off detailed analysis of the other components until later in the classyou will revisit this archive again once you finish the other modules. Be prepared to back up any claims with appropriate evidence.

You should be able to answer these questions once you have finished your analysis:

EMC Education Service

48

Step 1

Action Is the hit percentage unusually low for an open systems environment?

Are there any System or Device write pending issues?

Is there any evidence of data misalignment?

If there are any issues, what recommendations do you have for resolving them?

End of Lab 4 Part 5

EMC Education Service

49

Lab Exercise 5: Symmetrix Back-end


Purpose:

To use guided and free-format techniques to detect back-end issues.

Tasks:

In this lab, you will perform the following tasks:

Analyze an archive and look for issues related to the Symmetrix back end components Plan for the deployment of a new database on an existing Symmetrix Examine the effectiveness of prefetching

EMC Education Service

50

Lab 5: Part 1 Guided analysis

Performance Archive Used In This Part Email SRDF: Symmetrix/000284701058/daily/20040420d.btp

This archive was recorded at a site that uses Solaris hosts to support an Email application. The application is replicated to disaster recovery site using four SRDF RA1 ports. Another four RA2 ports receive data from the disaster recovery site. Step 1 Action Look at the Dir-DA>Disk Directors Utilization View.

Would you say that any of the directors is heavily utilized? _________ The average line is very close to the max line in this graph. What does that mean about the variance between the directorshow different are their individual utilizations? ___________________________________________________________

Look at the Dir-DA>All Disk Directors IOs per Sec View.

Do the DAs look balanced? ______________ Does the traffic on the DAs generally correlate with the front end traffic? Are the peak and valley times roughly the same? ______________

EMC Education Service

51

Step 3

Action Look at the Disks>Disks Utilization View.

What is the maximum utilization of any disk, and when does it occur? ______________

Use the Metrics tab to show a histogram of the Devices>total ios per sec for all active devices.

Try to use this display to detect groupings of devicesdevices involved in the same workload. Verify your guesses by displaying the Devices>total ios per sec in a normal time graph. If the devices are all part of the same workload, they will often have similar workload patterns.

Use the Metrics tab to show the Disks>total SCSI commands per sec

Are the disks well balanced? _________

EMC Education Service

52

Step 6

Action Pick the most heavily utilized disk, and plot the 6 measures which are summed to generate the total SCSI commands per sec measure all on the same graph. Do the same with the least heavily utilized disk.

What are the top two measures that make up most of the workload for the heavily utilized disk? ___________________________________________________________ What is the top measure that makes up the workload for the lightly utilized disk? _________________________________________________________________

Is XOR activity a large factor in the performance of any of the arrays disks? ________

Plot the Dir-DA>prefetched tracks per sec, Dir-DA>tracks not used per sec and Dir-DA>tracks used per sec on the same graph for one of the disk directors.

Is the prefetching activity constant or temporary? ______________________ When prefetching does trigger, is it generally effective? _________________ How can the tracks used per sec be higher than the prefetched tracks per sec? ______________________________________________________________

End of Lab 5 Part 1

EMC Education Service

53

Lab 5: Part 2 Auto design back-end performance

Performance Archive Used In This Part Automobile Design: Symmetrix/000284500972/revolving/200204181502.btp

This archive was taken from an array using 72GB RAID-1 drives. To help visualize the effects of adding a new database application, a load operation (write, read back and verify) was performed while this archive was created to capture the performance. Your analysis will explain the effects to the IT group and help them plan for future applications to be added to the array

Analyze the back-end performance of the Automobile Design archive. You have already examined the frontend and cache performance of this archive, now examine any issue (good or bad performance) related to the topics discussed in this module. Refer to the back-end road map for guidance. Be prepared to back up any claims with appropriate evidence.

Here are some questions you should be able to answer once you have done your analysis:

EMC Education Service

54

Step 1

Action Are any back end directors and disks being over utilized?

Is the traffic on the back end directors and disks balanced?

Are there significant back end resources for additional traffic?

If there are any issues, what recommendations do you have for resolving them?

End of Lab 5 Part 2

EMC Education Service

55

Lab 5: Part 3 Prefetch analysis


Performance Archive Used In This Part SAP/Oracle: Symmetrix/000187401250/interval/20040707.btp

This exercise uses an earlier snapshot of the SAP/ORACLE archive to investigate some basic prefetch metrics. The effectiveness of prefetching in the late night backup job is of the most concern.

Step 1

Action Compare the system-wide total ios per sec measure with the system-wide prefetched tracks per sec measure.

Are any changes in the measures evident during the late night backup period? What do these changes signify?

What is the approximate hit rate during this time?

EMC Education Service

56

Step 2

Action Compare the system-wide Prefetched tracks per second measure with the system-wide Tracks not used measure. Since the second of these is not available at the system level, you will have to plot it at the Dir-DA level. You will find it easiest to create a custom view that sums the measures for all disk directors to produce just one Tracks not used line.

Does the difference between prefetched tracks and tracks not used indicate an improvement in the effectiveness of prefetching during the backup job?

What is the approximate percentage of tracks not used to prefetched tracks during the backup job?

End of Lab 5 Part 3

EMC Education Service

57

Lab 5: Part 4 24/7 Back-end performance


Performance Archive Used In This Part 24/7: Symmetrix/000190100720/interval/20080312.btp

Analyze the overall back end performance of this archive. Thoroughly examine any issue (good or bad performance) related to the topics discussed in this module. Be prepared to back up any claims with appropriate evidence.

You might have thoroughly examined the back end performance of this archive during previous exercises in this Lab. If so, just make sure you have not missed anything.

Here are some questions you should be able to answer once you have done your analysis:

Step 1

Action Are any back end directors and disks being over utilized?

Is the traffic on the back end directors and disks balanced?

EMC Education Service

58

Step 3

Action Are back-end resources capable of handling additional applications?

Do you have any recommendations of how best to add new applications?

If there are any issues, what recommendations do you have for resolving them?

End of Lab 5 Part 4 End of Lab Exercise 5

EMC Education Service

59

Lab Exercise 6: Business Continuity


Purpose:

To use guided and free-format techniques to detect business continuity issues.

Tasks:

In this lab, you will perform the following tasks:

Examine SRDF/S communications Investigate Business Continuance issues and detect effects on overall Symmetrix performance

EMC Education Service

60

Lab 6: Part 1 Guided Analysis


Performance Archive Used In This Part Email SRDF: Symmetrix/000284701058/daily/20040420d.btp

This exercise covers the Email SRDF archive. This lab will simply ask a lot of questions about the archive; use Performance Manager to get the answers. This 8730 archive was recorded at a site that uses Solaris hosts to support an Email application. The application is replicated to disaster recovery site using four SRDF RA1 ports. Another four RA2 ports receive data from the disaster recovery site. They are using SRDF/S to replicate all data.

Step 1

Action View the I/O traffic on the four RA1 ports in one graph, and the I/O traffic on the four RA2 ports in another graph.

Do the ports within an RA group appear balanced? ____________

Do the two groups appear to be carrying different traffic loads? ____________

EMC Education Service

61

Step 2

Action Compare the I/O traffic on the front-end ports to the traffic on the two RA groups.

Which front-end ports are responsible for the outgoing SRDF traffic? _________________ What is the average I/O size on these ports? _____________________________ What is the approximate write percentage of those ports? ___________________________

Does the sum of the I/Os on the two front-end ports appear to be equal to the sum of the I/Os on the RA group? ___________ What would you expect to see on arrays using SRDF/A mode? ________________________________________________________________________

Compare the I/O traffic of a few of the most-utilized devices to the traffic on the frontend ports you considered in the previous step.

Can you quickly identify devices that are carrying the outgoing SRDF load?

End of Lab 6 Part 1

EMC Education Service

62

Lab 6: Part 2 Unexpected Night Delays


Performance Archive Used In This Part Symmetrix/000284500356/daily/20021214d.btp Symmetrix/000284500356/daily/20021215d.btp

In this Part, you will be investigating application slowdowns that are occurring late in the night. Activity from around 18:00 to 03:00 is noticeably slower than the rest of the day. Investigate the archive and answer the questions.

Step 1

Action What event is most likely causing the change in response time?

What effects are these events likely to be having on the Symmetrix? Support your answer with evidence whenever possible.

What software solution[s] might help reduce the negative performance effects?

End of Lab 6 Part 2 End of Lab Exercise 6

EMC Education Service

63

You might also like