Uc434s.f.00 Stu - Part2 PDF

Accelerated SAN Essentials
UC434S F.00
Student guide
2 of 2
Use of this material to deliver training without prior written permission from HP is prohibited.

UC434S F.00
Student guide
2 of 2
Use of this material to deliver training without prior written permission from HP is prohibited.
Copyright 2010 Hewlett-Packard Development Company, L.P.

The information contained herein is subject to change without notice. The only warranties for HP
products and services are set forth in the express warranty statements accompanying such products
and services. Nothing herein should be construed as constituting an additional warranty. HP shall
not be liable for technical or editorial errors or omissions contained herein.
This is an HP copyrighted work that may not be reproduced without the written permission of HP.
You may not use these materials to deliver training to any person outside of your organization
without the written permission of HP.
Microsoft, Windows, Windows XP, and Windows Vista are U.S. registered trademarks of
Microsoft Corporation.
UNIX is a registered trademark of The Open Group.
Export Compliance Agreement
Export Requirements. You may not export or re-export products subject to this agreement in violation
of any applicable laws or regulations.
Without limiting the generality of the foregoing, products subject to this agreement may not be
exported, re-exported, otherwise transferred to or within (or to a national or resident of) countries
under U.S. economic embargo and/or sanction including the following countries:
Cuba, Iran, North Korea, Sudan and Syria.
This list is subject to change.
In addition, products subject to this agreement may not be exported, re-exported, or otherwise
transferred to persons or entities listed on the U.S. Department of Commerce Denied Persons List;
U.S. Department of Commerce Entity List (15 CFR 744, Supplement 4); U.S. Treasury Department
Designated/Blocked Nationals exclusion list; or U.S. State Department Debarred Parties List; or to
parties directly or indirectly involved in the development or production of nuclear, chemical, or
biological weapons, missiles, rocket systems, or unmanned air vehicles as specified in the U.S.
Export Administration Regulations (15 CFR 744); or to parties directly or indirectly involved in the
financing, commission or support of terrorist activities.
By accepting this agreement you confirm that you are not located in (or a national or resident of)
any country under U.S. embargo or sanction; not identified on any U.S. Department of Commerce
Denied Persons List, Entity List, US State Department Debarred Parties List or Treasury Department
Designated Nationals exclusion list; not directly or indirectly involved in the development or
production of nuclear, chemical, biological weapons, missiles, rocket systems, or unmanned air
vehicles as specified in the U.S. Export Administration Regulations (15 CFR 744), and not directly
or indirectly involved in the financing, commission or support of terrorist activities.
Printed in USA
Student guide
December 2010
Contents
Module 8 - iSCSI
Objectives ................................................................................................. 8 - 1
IP storage .................................................................................................. 8 - 2
IP storage protocols .................................................................................... 8 - 4
Overview of iSCSI protocol .......................................................................... 8 - 6
iSCSI maps SCSI onto a network ............................................................ 8 - 7
Overview of iSCSI protocol ................................................................... 8 - 8
iSCSI/FC SAN ..........................................................................................8 - 10
iSCSI Stack ...............................................................................................8 - 12
iSCSI encapsulation ................................................................................... 8 - 13
iSCSI Packet.............................................................................................. 8 - 14
iSCSI Host Driver ....................................................................................... 8 - 15
iSCSI initiators...........................................................................................8 - 16
iSCSI Name Support..................................................................................8 - 17
iSCSI Name Structure (1 of 2) .....................................................................8 - 19
iSCSI Name Structure (2 of 2) .....................................................................8 - 21
iSCSI name examples ............................................................................... 8 - 22
iSNS........................................................................................................8 - 23
iSCSI target discovery ............................................................................... 8 - 25
iSCSI target discovery example ...................................................................8 - 26
iSCSI operations........................................................................................8 - 27
iSCSI authentication .................................................................................. 8 - 28
iSCSI CHAP examples............................................................................... 8 - 29
IP Security ............................................................................................... 8 - 30
iSCSI advantages & disadvantages.............................................................. 8 - 31
P4000 HP StorageWorks iSCSI SAN .....................................................8 - 32
HP StorageWorks iSCSI SAN Recommended architecture........................ 8 - 33
Centralized Management Console (CMC) ............................................. 8 - 34
CMC Navigation ............................................................................... 8 - 35
Hierarchical Structure.......................................................................... 8 - 36
HP StorageWorks SAN Concepts ..........................................................8 - 37
Network RAID data mirroring .............................................................. 8 - 38
Configuring an HP StorageWorks SAN ................................................. 8 - 40
Discovering Storage Nodes ..................................................................8 - 41
Creating a Management Group............................................................8 - 42
Creating a Cluster .............................................................................. 8 - 43
Creating a Volume ............................................................................. 8 - 44
Creating a Server............................................................................ 8 - 45
Connecting a Volume to a Server ......................................................... 8 - 46
Final Result .........................................................................................8 - 47
Lab activity .............................................................................................. 8 - 48
UC434S F.00
2010 Hewlett-Packard Development Company, L.P.
Module 9 - SAN extension

Objectives ................................................................................................. 9 - 1
What is SAN extension? ............................................................................. 9 - 2
Why extend the SAN? ................................................................................ 9 - 3
HP Supported SAN extension technologies .................................................... 9 - 4
SAN extension distance summary .............................................................. 9 - 5
Long Wave Transceivers .............................................................................. 9 - 6
Coarse Wave Division Multiplexing ............................................................. 9 - 8
Dense Wave Division Multiplexing................................................................ 9 - 9
FCIP......................................................................................................... 9 - 11
FCIP Protocol Mapping............................................................................9 - 12
Fibre Channel over IP................................................................................. 9 - 13
Fibre Channel over IP................................................................................. 9 - 14
FCIP performance......................................................................................9 - 16
Network speeds ........................................................................................9 - 18
Basic SCIS WRITE OPERATION...................................................................9 - 21
Brocade Fastwrite ......................................................................................9 - 23
Brocade FCIP Fastwrite + Tape pipelining.................................................... 9 - 25
Cisco Write Acceleration ............................................................................9 - 27
FCIP Compression .................................................................................... 9 - 29
IP network considerations .......................................................................... 9 - 30
IP network best practices ............................................................................ 9 - 31
FCIP Security.............................................................................................9 - 32
FCIP encryption ........................................................................................ 9 - 33
FCIP advantages ...................................................................................... 9 - 35
FCIP hardware ......................................................................................... 9 - 36
Fibre Channel routing overview ...................................................................9 - 37
Fabric and VSAN independence ................................................................ 9 - 39
SAN scaling .............................................................................................9 - 41
Fabric services limits ................................................................................. 9 - 43
Scaling by Routing.................................................................................... 9 - 45
Fibre Channel routing implementations........................................................ 9 - 46
B-Series and C-Series routing differences ..................................................... 9 - 48
Basic MP Router configuration .....................................................................9 - 49
Fibre Channel routing techniques H-Series switch ....................................... 9 - 50
Routing through an IP network..................................................................... 9 - 51
Five-fabric configuration with FC-IP...............................................................9 - 52
SAN island consolidation ...........................................................................9 - 53
Integration of Fibre Channel routing and FCIP ...............................................9 - 55
Six-fabric configuration.............................................................................. 9 - 56
Tape backup consolidation .........................................................................9 - 57
Broadcast Zones....................................................................................... 9 - 58
ii
UC434S F.00
Contents
Module 10 - FCoE/CEE
Objectives ................................................................................................10 - 1
FCoE (Fibre Channel over Ethernet) .......................................................10 - 2
CEE (Converged Enhanced Ethernet ......................................................10 - 2
FCoE/CEE................................................................................................10 - 4
FCoE .......................................................................................................10 - 7
FcoE Terminology ......................................................................................10 - 9
FCoE integrated with FC San fabric ........................................................... 10 - 10
OSI, FCoE and FC stacks...........................................................................10 - 11
FCoE encapsulation ................................................................................. 10 - 12
Lossless Ethernet ...................................................................................... 10 - 14
HP Converged network switches offerings ................................................... 10 - 15
Converged Network Adapters (CANs) ....................................................... 10 - 18
Ethernet Overview ...................................................................................10 - 20
CEE Map ...............................................................................................10 - 22
DCBX (Data Center Bridging eXchange Protocol)......................................... 10 - 23
VLAN Membership ..................................................................................10 - 25
Minimum CEE configuration to allow FCoE traffic flow ................................. 10 - 27
FCIP, ISCSI & FCoE ..................................................................................10 - 29
Storage Support ...................................................................................... 10 - 31
Operating System Support ........................................................................ 10 - 32
Module 11 - SAN Management

SAN Management .................................................................................... 11 - 1
Objectives ................................................................................................ 11 - 1
Storage management tasks ......................................................................... 11 - 2
Storage Resource Management ................................................................... 11 - 5
SAN management categories ..................................................................... 11 - 6
SAN management ..................................................................................... 11 - 8
SAN performance management .................................................................. 11 - 9
Storage capacity management................................................................... 11 - 11
SMI-S ......................................................................................................11 - 13
Implementing SMI-S ..................................................................................11 - 14
Storage Essentials .....................................................................................11 - 17
Storage Essentials Enterprise Edition plug-ins ................................................11 - 19
Storage Essentials Enterprise Edition home page ......................................... 11 - 20
Description of base components .................................................................11 - 21
System Manager ..................................................................................... 11 - 22
Capacity Manager .................................................................................. 11 - 23
Performance Manager ............................................................................. 11 - 24
Application Viewer .................................................................................. 11 - 25
Policy Manager ....................................................................................... 11 - 26
Event Manager ....................................................................................... 11 - 27
Report Optimizer ..................................................................................... 11 - 28
Description of plug-ins for both editions ...................................................... 11 - 30
UC434S F.00
iii
Database Viewer ......................................................................................11 - 31

Exchange Viewer..................................................................................... 11 - 32
File System Viewer ................................................................................... 11 - 33
Backup Manager..................................................................................... 11 - 34
HP StorageWorks Fabric Manager ............................................................ 11 - 35
Key features and functions (1 of 2) ............................................................. 11 - 37
Key features and functions (2 of 2)............................................................. 11 - 39
Graphical user interface ............................................................................11 - 41
Firmware upgrade ................................................................................... 11 - 43
Configuration upload and download ......................................................... 11 - 45
HP Data Center Fabric Manager (DCFM) ................................................... 11 - 46
Utilize Administration Time........................................................................ 11 - 47
Cisco Fabric Manager overview ................................................................ 11 - 49
Fabric Manager ...................................................................................... 11 - 50
Device Manager ......................................................................................11 - 51
Lab activity ............................................................................................. 11 - 54
Module 12 - SAN Security

Objectives ................................................................................................12 - 1
Security in a SAN......................................................................................12 - 2
Basic security model ..................................................................................12 - 3
Security domains .......................................................................................12 - 5
Attacks and exposures ...............................................................................12 - 7
Mitigation of risk .......................................................................................12 - 8
SAN security access points ....................................................................... 12 - 10
Storage security model ..............................................................................12 - 11
Data security........................................................................................... 12 - 12
Management security............................................................................... 12 - 19
Local Authentication on a switch ......................................................... 12 - 22
Role-Based Access Control (RBAC) B-Series........................................... 12 - 23
Role-Based Access Control (RBAC) C-Series .......................................... 12 - 26
RADIUS Authentication............................................................................. 12 - 27
SAN security practices ............................................................................. 12 - 29
Planning SAN security prevention .............................................................. 12 - 30
Data path and management path security in practice .................................. 12 - 31
Storage security in an enterprise environment .............................................. 12 - 32
Security in practice .................................................................................. 12 - 35
Authentication ......................................................................................... 12 - 36
FCIP encryption and Data encryption at rest................................................ 12 - 38
Lab activity ............................................................................................. 12 - 41
Module 13 - Data protection

Objectives ................................................................................................ 13 - 1
Data Protection.......................................................................................... 13 - 2
Challenges in Data Protection ..................................................................... 13 - 3
iv
UC434S F.00
Contents
Not all data are equal ............................................................................... 13 - 4

Recovery operations................................................................................... 13 - 5
Protection and recovery methods ................................................................. 13 - 6
Data Protection Technologies ...................................................................... 13 - 7
Direct backup tape.................................................................................. 13 - 9
Centralized server backup ........................................................................ 13 - 10
Automated centralized backup ...................................................................13 - 11
Centralized SAN backup .......................................................................... 13 - 12
Tape libraries .......................................................................................... 13 - 13
Zoning for backup ................................................................................... 13 - 14
Backup performance considerations........................................................... 13 - 16
Virtual Tape Libraries ............................................................................... 13 - 18
VTL in practice ........................................................................................ 13 - 20
Disk to Tape............................................................................................ 13 - 22
Data replication ...................................................................................... 13 - 23
Split-mirror backup concept ...................................................................... 13 - 24
Snapshot backup concept......................................................................... 13 - 26
De-Duplication ........................................................................................ 13 - 27
How hash based chunking works .............................................................. 13 - 32
How hash-based chunking performs restores ............................................... 13 - 34
Pros & Cons of HP Dynamic Deduplication........................................... 13 - 36
Enterprise Deployment with replication across remote and branch offices
back to data centers .......................................................................... 13 - 37
Hp Storage Works Disk to Disk and Virtual Library portfolio with
de-duplication................................................................................... 13 - 38
Remote replication ................................................................................... 13 - 39
HP StorageWorks Continuous Access EVA (CA EVA) .................................... 13 - 40
Synchronous replication (1 of 2) ................................................................ 13 - 41
Synchronous replication (2 of 2) ................................................................ 13 - 42
Asynchronous replication (1 of 2)............................................................... 13 - 43
Asynchronous replication (2 of 2) .............................................................. 13 - 44
Comparing replication modes ................................................................... 13 - 45
HP StorageWorks Storage Mirroring .......................................................... 13 - 47
SWSM mirroring full.............................................................................. 13 - 48
SWSM mirroring file difference............................................................... 13 - 49
Lab activity ............................................................................................. 13 - 50
Module 14 - Performance
Objectives ................................................................................................ 14 - 1
SAN performance objectives....................................................................... 14 - 2
Performance factors ................................................................................... 14 - 4
Response time ........................................................................................... 14 - 7
Bus utilization............................................................................................ 14 - 8
Device utilization ....................................................................................... 14 - 9
SAN performance Considerations...............................................................14 - 11
UC434S F.00
Latencies ................................................................................................ 14 - 13
ISL oversubscription.................................................................................. 14 - 15
Hop latency............................................................................................ 14 - 17
Data Priority Quality of Service............................................................... 14 - 19
Device attachment points .......................................................................... 14 - 21
Place fastest switches in the core................................................................ 14 - 22
Distance considerations............................................................................ 14 - 24
Maintaining performance in an extended SAN beyond 5 or 10km ................ 14 - 25
Distributed fabrics.................................................................................... 14 - 26
Long distance link modes ......................................................................... 14 - 27
Extended distance topology ...................................................................... 14 - 28
Performance Guidelines within the SAN ..................................................... 14 - 29
Determining the required bandwidth .......................................................... 14 - 30
Drive selection and performance ............................................................... 14 - 32
RAID and RAID selection .......................................................................... 14 - 34
RAID level efficiency................................................................................. 14 - 36
Disk Performance..................................................................................... 14 - 37
Planning a disk system ............................................................................. 14 - 38
Data caching technologies ....................................................................... 14 - 41
Write-back caching ................................................................................. 14 - 43
Write-back cache benefits ........................................................................ 14 - 45
Effects of cache ....................................................................................... 14 - 47
Application effects on performance............................................................ 14 - 49
Environment profiling ............................................................................... 14 - 50
Large sequential read environment............................................................. 14 - 51
Server Application ................................................................................... 14 - 52
Improving performance ............................................................................ 14 - 56
Comparing VRAID1 and VRAID5 .............................................................. 14 - 57
Safe IOPs calculator for production disk group ............................................ 14 - 59
Safe IOPs calculator Microsoft version.................................................... 14 - 61
EVAPerf.................................................................................................. 14 - 62
End to End monitoring.............................................................................. 14 - 65
Top talker ............................................................................................... 14 - 66
Lab activity ............................................................................................. 14 - 68
vi
UC434S F.00
iSCSI
Module 8
Objectives
UC434S F.00
8 -1
IP storage
IP storage
IP storage can combine the following functions on a single
enterprise network:
Storage
Data sharing
Web access
Device management using SNMP
E-mail
Voice and video transmission
With many of the benefits that Fibre Channel SANs
already give to us.
The amount of data stored has been doubling every year and this has been
attributed to the phenomenal growth in software applications, such as on-line
transactions, e-mail, and the development of complex e-commerce applications. The
Internet and corporate intranets drive this growth to an extent where there is an
almost mandatory requirement for continuous availability of information in the
corporate e-business world. The net effect of this trend has been the duplication of
on-line copies of this monumental quantity of data. This increasing appetite to
consume disk storage has been met by the disk drive industry to double the capacity
of hard disk drives and to reduce the price of storage.
The pervasiveness of the Internet Protocol (IP) through the unprecedented growth of
the Internet and the increasing demand of disk storage has led to the question as to
whether or not it is possible to use TCP/IP, the networking technology of Ethernet
LANs and the Internet, for use in disk storage.
Such an approach can facilitate a single network:
Storage
Data sharing
Web access
Device management using SNMP
E-mail
8 -2
Voice and video transmission

UC434S F.00
iSCSI
IP networking is based on design considerations different from those of storage

concepts. While the TCP/IP protocol is software-based and geared towards
unsolicited packets, storage protocols are hardware-based and are centered on
solicited packets. A storage networking protocol must leverage the TCP/IP stack
without any modification and maintain high performance. The goal is to merge the
two concepts and provide the performance of a specialized storage protocol like
SCSI. This amalgamation of storage area networks (SANs) and IP is the driving force
for using IP-based networks to transport block storage traffic and is referred to as IP
storage.
Benefits
IP storage has emerged in recent years as networked storage requirements have
grown and IP has become firmly established as the predominant general purpose
networking protocol. The following are some benefits provided by IP storage:
Addresses the difficulties of managing burgeoning storage resources.
Facilitates the storing, accessing, protection, and management of mission-critical

data.
Provides protection of data by allowing redundant paths between host and

storage devices.
Enables remote mirroring solutions for disaster recovery.
Allows backups to be performed over the IP storage with minimal impact on

application servers or the host network.
Allows storage to be consolidated, which reduces management complexity.
Centralized management of a consolidated storage pool can be more efficient than

managing separate direct-attached storage subsystems.
With an ever-growing development base behind IP, the existing quality of service,
link prioritization, and security protocols that are available for IP networks continues
to drive technology.
IP is relatively inexpensive, because it runs over commodity sub-networking
technologies such as Ethernet.
Enables block storage over IP-based networks and provides easy access to storage
over long distances.
A single networking technology for the LAN and SAN is compelling such that IT
departments do not have to maintain equipment, technical staff, and expertise in
both the IP and FC technologies.
UC434S F.00
8 -3
IP storage protocols
IP storage protocols
iSCSI
iFCP
FCIP
Devices
Fabric
Services
Understanding IP storage protocols

An appreciation of the IP protocol is necessary in order to understand IP storage
protocols. The network layer protocol which lies below the transport layer is known
as the Internet Protocol (IP). It is responsible for transferring data from one host to
another, using various routing algorithms. Layers above the network layer fragment a
data stream into chunks of a predetermined size, known as packets or datagrams.
The datagrams are then sequentially passed to the IP network layer.
The purpose of the IP layer is to route such packets to the target destination. IP
packets comprise an IP header, together with the higher level TCP protocol and the
application datagram. IP knows nothing about the TCP and datagram contents. Prior
to transmitting data, the network layer could further subdivide the data into smaller
packets to facilitate transmission. On receipt at the destination, the packets are
reassembled into the original datagram by the network layer.
The transfer of block-level storage data across a networked topology is not a new
concept and is achieved currently through SANs using the Fibre Channel technology.
Transferring data through the use of IP is not a new concept. Protocols such as
Common Internet File System (CIFS) and Network File System (NFS) have been used
to access storage data over IP networks for several years. The difference between
these existing protocols and the IP storage protocols lie in their respective approach
to the means of accessing data. CIFS and NFS access data at the file-level while IP
storage protocols access data at the block-level.
8 -4
UC434S F.00
iSCSI
The following three IP storage encapsulation protocols have been defined:
iSCSI
iSCSI is defined as a SCSI network transport protocol that operates with TCP as the
underlying layer to provide a reliable transport with guaranteed in-order delivery.
iSCSI encapsulates SCSI protocols into a TCP/IP frame, so that storage controllers
can be attached to IP networks.
Fibre Channel over TCP/IP

The purpose of the Fibre Channel Over TCI/IP (FCIP) protocol is to transport Fibre
Channel frames over an IP infrastructure. FCIP provides the mechanisms facilitating
islands of Fibre Channel SANs to be interconnected over IP-based networks to form a
single, unified Fibre Channel SAN fabric or separate managed Fibre Channel SANs.
The extended Fibre Channel SAN fabric continues to use standard Fibre Channel
addressing. IP tunnels are set up between FCIP end points. Once these tunnels are in
place, Fibre Channel devices view these extended links as standard Fibre Channel
links and use Fibre Channel addressing.
FCIP tunnels
FCIP tunnels are used to pass Fibre channel I/O through an IP network. FCIP tunnels
are built on a physical connection between two peer switches or blades.
VE_ports and VEX_ports

VE_ports and VEX_ports are virtual E_ports. VE_Ports are used to create interswitch
links (ISLs). If VE_Ports are used on both ends of an FCIP tunnel, the fabrics
connected by the tunnel are merged. VEX_Ports enable interfabric links (IFLs). If a
VEX_Port is on one end of an FCIP tunnel, the fabrics connected by the tunnel are
not merged. The other end of the tunnel must be defined as a VE_Port. VEX_Ports
cannot be used in pairs
Internet Fibre Channel Protocol

Internet Fibre Channel Protocol (iFCP) encapsulates Fibre Channel frames to be sent
over the IP infrastructure just like the FCIP protocol. Both protocols use a common
Fibre Channel encapsulation format. The principal difference between the two
protocols lies in the addressing schemes. The FCIP protocol establishes point-to-point
tunnels that can be used to connect two Fibre Channel SANs together using Ethernet
to create a single, larger SAN. In contrast, iFCP is a gateway-to-gateway protocol
that combines Fibre Channel and IP addressing to allow the Fibre Channel frames to
be routed to the appropriate destination address. Unlike the addressing scheme of
the FCIP protocol, the current iFCP addressing scheme allows each interconnected
SAN to retain its own independent name space.
UC434S F.00
8 -5
Overview of iSCSI protocol

What is it?
iSCSI is an IETF SCSI transport protocol for mapping of
block-oriented storage data over TCP/IP networks.
The iSCSI protocol enables universal access to storage
devices and Storage Area Networks (SANs) over
standard Ethernet-based TCP/IP networks
These networks may be dedicated networks or may be
shared with traditional Ethernet applications.
What is it?
iSCSI is an IETF SCSI transport protocol for mapping of block-oriented storage data
over TCP/IP networks.
The iSCSI protocol enables universal access to storage devices and Storage Area
Networks (SANs) over standard Ethernet-based TCP/IP networks
These networks may be dedicated networks or may be shared with traditional
Ethernet applications.
8 -6
UC434S F.00
iSCSI
iSCSI maps SCSI onto a network
SCSI is a long-established protocol for connecting disks to computers. All

common operating systems contain drivers and logic for SCSI devices. By
replacing the SCSI hardware driver with a SAN initiator, iSCSI creates a
connection into the target SAN system. The user system sees the iSCSI
connection as if it was a normal SCSI disk, so no further modifications are
required to the accessing system or applications.
UC434S F.00
8 -7

Why was it created?
The SCSI protocol has been mapped over various
transports, including Parallel SCSI, IPI, IEEE-1394 (firewire)
and Fibre Channel. These transports are I/O specific and
have limited distance capabilities.
The iSCSI protocol is a means of transporting of the SCSI

packets over TCP/IP, providing for an interoperable
solution which can take advantage of existing Internet
infrastructure, Internet management facilities and address
distance limitations.
Why was it created?

The SCSI protocol has been mapped over various transports, including Parallel SCSI,
IPI, IEEE-1394 (firewire) and Fibre Channel. These transports are I/O specific and
have limited distance capabilities.
The iSCSI protocol is a means of transporting of the SCSI packets over TCP/IP,
providing for an interoperable solution which can take advantage of existing Internet
infrastructure, Internet management facilities and address distance limitations.
The iSCSI protocol uses TCP/IP for its data transfer. Unlike other network storage
protocols, such as Fibre Channel (which is the foundation of most SANs), it requires
only the simple and ubiquitous Ethernet interface (or any other TCP/IP-capable
network) to operate. This enables low-cost centralization of storage without all of the
usual expense and incompatibility normally associated with Fibre Channel storage
area networks.
Critics of iSCSI expect worse performance than Fibre Channel due to the overhead
added by the TCP/IP protocol to the communication between client and storage.
However new techniques like TCP Offload Engine (TOE) help in reducing this
overhead. Tests have shown excellent performance of iSCSI SANs, whether TOEs or
plain Gigabit Ethernet NICs were used. In fact, in modern high-performance servers,
a plain NIC with efficient network driver code can outperform a TOE card because
fewer interrupts and DMA memory transfers are required. Initial iSCSI solutions are
based on a software stack. The iSCSI market is growing steadily, and should improve
in performance and usability as more organizations deploy Gigabit and 10 Gigabit
networks, and manufacturers integrate iSCSI support into their operating systems,
8 -8
UC434S F.00
iSCSI
SAN products and storage subsystems. iSCSI became even more interesting once
Ethernet started to support higher speeds than Fibre Channel.
UC434S F.00
8 -9
iSCSI/FC SAN
iSCSI/FC SAN
Storage device
Server with IP
storage adapter
Server with IP
storage adapter
Server with FC
storage adapter
Storage device
Storage device
Server with IP
storage adapter
Storage device
Fibre Channel has provided the principal means for building SANs because of the
rich features of high performance, connectivity, and ability to support block-oriented
storage protocols. The high throughput is achieved by assigning much of the protocol
processing to hardware. Fibre Channel overcomes several scalability issues inherent
in SCSI by creating a switched network fabric infrastructure that extends Fibre
Channel operating distance between 10 and 20km and overcomes device count
limitations.
IT managers are concerned about sharing storage traffic and data traffic within a
common IP network backbone. The principal concern is that such sharing could lead
to congestion bottlenecks. While combining both messaging and storage traffic on a
single network is possible, a more practical solution is to segment the IP network
infrastructure and move storage and data traffic via different paths. This approach
enables customers to protect the investment in IP networking and maximize the
efficiencies of moving both types of traffic over a common infrastructure.
The common IP storage network technology for both iSCSI and Fibre Channel
connected devices provides the following capabilities and benefits:
8 -10
Universal storage access across an IP network regardless of the interconnect (for

example, iSCSI and Fibre Channel)
Access to existing Fibre Channel SANs and a migration path to an IP

environment
Use of Fibre Channel end systems with proven performance and relative stability
UC434S F.00
iSCSI
High-end storage and SAN expansion for iSCSI servers
Pooling iSCSI SANs, Fibre Channel SANs, and network attached storage (NAS)
resources over a common IP network for a viable long-term storage strategy
Improved storage manageability and high availability of storage resources
A core SAN fabric that is IP-based
These environments can provide application support for:
Local and remote backup over an IP network.
Storage virtualization across a common pool of heterogeneous storage

resources.
Peer-to-peer copy.
Disaster recovery and high availability.
Mirroring across heterogeneous SANs.
UC434S F.00
8 -11
iSCSI Stack
iSCSI Stack
iSCSI
OSI Model
Application
Presentation
Session
SCSI Device-Type
Commands
SCSI Applications
(File Systems,
Databases)
SCSI Block
Commands
SCSI Generic
Commands
SCSI Commands,
Data, and Status
SCSI Transport
Protocols
iSCSI
Transport
TCP
Network
IP
Data Link
Physical
Ethernet
iSCSI uses TCP/IP for reliable data transmission over potentially unreliable networks.
The iSCSI layer interfaces to the operating system standard SCSI set and includes
encapsulated SCSI commands, data and status reporting capability. When the
operating system or application requires a data write operation, the SCSI CDB must
be encapsulated for transport over a serial gigabit link and delivered to the target.
The iSCSI protocol monitors the block data transfer and validates completion of the
I/O operation. This occurs over one or more TCP connections between initiator and
target. In practical applications, an initiator can have multiple target resources over
an IP network and consequently, multiple concurrent TCP connections are active.
The iSCSI protocol maps the SCSI Remote Procedure Call model to the TCP/IP
protocol and provides a conceptual layer completely independent of the SCSI CDB
information. SCSI commands are transported by iSCSI request and SCSI response
and status are handled by iSCSI responses. iSCSI protocol tasks are then carried by
this same iSCSI request and response mechanism. Following the pattern of the SCSI
protocol, iSCSI employs the concepts of initiator, target, and communication
messages called protocol data units (PDUs). Likewise, iSCSI transfer direction is
defined respectively to the initiator. As a means to improve performance, iSCSI
allows a phase collapse that provides a command or response and its associated
data to be sent in a single iSCSI PDU.
8 -12
UC434S F.00
iSCSI
iSCSI encapsulation
iSCSI encapsulation
Layer 2
(Ethernet)
Physical
addressing
information
IP
Header
TCP
Header
iSCSI
Header
SCSI Commands and Data
Provides error-correction,
sequencing of packet, and
identifies application using the
service
Indicates how to extract SCSI

commands and data
Network address and

routing information used
in routing the packet
The iSCSI standard stipulates that the protocol must not require modification to the
current IP and Ethernet infrastructure to support storage traffic. The iSCSI protocol
standard must allow implementations to equal or improve on the current state of the
art for SCSI interconnects. The iSCSI protocol:
Must provide low delay communications.
Must provide high bandwidth and bandwidth aggregation.
Must have low host CPU utilizations, equal to or better than current technology.
Must be possible to build I/O adapters handling the entire SCSI task.
Must permit zero-copy memory architectures.
Must not impose complex operations on host software.
Must be cost competitive with alternative storage networking technologies.
UC434S F.00
8 -13
iSCSI Packet
iSCSI Packet
46 1500 bytes
Ethernet header
Preamble
Destination
Source
Type
IP
TCP
Data
CRC
Well known TCP ports

21
ftp
23
telnet
25
smtp
80
http
3260
iSCSI Protocol Data Unit (PDU)

TCP Header
iSCSI
I Opcode
Header
Length
LUN
Source Port
Destination Port
Sequence Number
Reserved
Checksum
Opcode Specific Fields

Data Segment Length
or Opcode-specific fields
Initiator
Opcode
Acknowledgment Number
Offset
Window
Task Tag
Specific Fields
Data
Field
Urgent pointer
Options and padding
The basic system model for iSCSI is that of an extended virtual cable, connecting a
SCSI initiator device to a SCSI target device. Both the iSCSI initiator and iSCSI target
are identified by their IP addresses which are carried within the iSCSI packet header.
8 -14
UC434S F.00
iSCSI
iSCSI Host Driver

iSCSI Host Driver
HOST
Resides on the host and provides host-tostorage connectivity over an IP network

Uses the hosts existing TCP/IP stack, network
drivers and network interface card(s) (NIC) to
provide the same functions as native SCSI
drivers and Host Bus Adapter (HBA) cards
Functions as a transport for SCSI commands
and responses between the host and the iSCSI
target on an IP network.
Applications
File System
Block Device
SCSI Generic
iSCSI
Driver
TCP/IP Stack
NIC Driver Adapter Driver
NIC Adapter
FC/iSCSI bridge
ARRAY
Direct
connect
or
SAN
SCSI/TCP Server
SCSI Driver TCP/IP Driver
FC HBA
GigE NIC
IP Network
SCSI Adapter
(HBA)
Direct
Attached
Storage
Array
Resides on the host and provides host-to-storage connectivity over an IP network
Uses the hosts existing TCP/IP stack, network drivers and network interface
card(s) (NIC) to provide the same functions as native SCSI drivers and Host Bus
Adapter (HBA) cards
Functions as a transport for SCSI commands and responses between the host
and the iSCSI target on an IP network.
UC434S F.00
8 -15
iSCSI initiators
iSCSI initiators
Good
8 -16
Better
Best
The IP host or iSCSI Initiator uses an iSCSI Driver to enable target resource
recognition & attachment to the iSCSI target over IP.
The iSCSI initiator is configured with the Gigabit Ethernet IP address of the iSCSI
port on the iSCSI target to transport SCSI requests and responses.
The iSCSI initiator sees the storage resources (LUNs) as if they were local blocklevel drives attached directly to the server.
UC434S F.00
iSCSI
iSCSI Name Support

iSCSI Name Support
iSCSI names are:
used for identification
used for authentication
enable iSCSI resources to be managed regardless or
location
Each iSCSI initiator (and iSCSI target) must have an iSCSI
name.
iSCSI names:
can be up to 223 bytes in length
are globally unique
do not imply a location or address
May be registered with iSNS
iSCSI Name Support

iSCSI implements a client-server model between disk targets and initiating hosts and
adheres to the SCSI Architectural Model (SAM-2). Because both iSCSI targets and
initiators are elements on an IP network, the clients and servers have a Network
Entity identity that is equivalent to assigned IP addresses. It is possible for the
Network Entity to contain one or more iSCSI Nodes.
The iSCSI Node object uniquely identifies a SCSI device within a Network Entity that
is accessible through the network. The Network Portal is an amalgamation of the
Node IP address and the TCP port number. The Network Entity object allows for
multiple iSCSI Nodes because a Network Entity might represent a gateway fronting
multiple initiators or targets. Each iSCSI Node is identified by a unique iSCSI name
that can be up to 255 bytes in length. A Domain Name Server (DNS) or other
resource locator implementations can parse this identifier. The 255-byte name length
also ensures globally unique names that can be formatted as required by the storage
administrator.
iSCSI names are:
used for identification
used for authentication
enable iSCSI resources to be managed regardless or location
UC434S F.00
8 -17
Each iSCSI initiator (and iSCSI target) must have an iSCSI name.
iSCSI names:
8 -18
can be up to 223 bytes in length
are globally unique
do not imply a location or address
May be registered with iSNS
UC434S F.00
iSCSI
iSCSI Name Structure (1 of 2)

The iSCSI name structure is fairly rigid and contains two parts: a type
designation followed by a unique name string.
The two type designators for iSCSI are:
iqn.
iSCSI qualified name
eui.
IEEE EUI-64 identifier in ACSII-encoded hexadecimal
An example of the iqn string is shown below:
Iqn.2003-02.com.hp:server3
This designator type consists of the following:
The string iqn followed by a dot (.)
A date code, in yyyy-mm format followed by another dot (.)
A reverse domain name, and
Optional colon (:) prefixed by string
iSCSI Name Structure

The combination of IP address and a TCP port generates a unique network address
for an iSCSI device. The separation of iSCSI names and iSCSI addresses ensures that
a storage device will have a unique identity in the network regardless of its location
in the network. Although the IP address plus the TCP port number will necessarily
change if a device is moved onto a different network segment, the iSCSI name will
travel with the device, allowing it to be rediscovered.
The iSCSI naming convention is meant to assist the discovery process and validate
device identity during iSCSI login between initiator and target. The potentially very
long 255-byte iSCSI name is not used for routing, which would place an
unreasonable burden on network parsing engines. Instead, after the IP address and
TCP port number are established for a specific iSCSI Node, only the IP address and
TCP port combination is required for storage transactions.
UC434S F.00
8 -19
T he iS CS I name s tructure is fairly rigid and contains two parts : a type des ignation
followed by a unique name s tring.
T he two type des ignators for iS CS I are:
iqn.
eui.
iSCSI qualified name

IE E E E UI- 64 identifier in ACS II- encoded hexadecimal
An example of the iqn s tring is s hown below:

Iqn.2003-02.com.hp:server3
T his designator type cons ists of the following:
T he s tring iqn followed by a dot (.)

A date code, in yyyy- mm format followed by another dot (.)
A reverse domain name, and
Optional colon (:) prefixed by s tring
iqn: specifies the use of the iSCSI qualified name as the authority.
Date Code: 2003-02 is the year and month on which the naming authority
acquired the domain name used in this iSCSI name. This is used to ensure that when
domain names are sold or transferred to another organization, iSCSI names
generated by these organizations will be unique.
com.hp is a reversed DNS name, and defines the organizational naming authority.
The owner of the DNS name hp.com has the sole right of use of this name as this
part of an iSCSI name, as well as the responsibility to keep the remainder of the
iSCSI name unique.
server3 was picked arbitrarily by hp.com to identify the server hosting the iSCSI
device. The owner of "hp.com" is responsible for keeping this structure unique
8 -20
UC434S F.00
iSCSI

There are also special rules that must be followed:
For those using only ASCII characters (U+0000 to
U+007F), the following characters are allowed:
ASCII dash character ('-' = U+002d)
ASCII dot character ('.' = U+002e)

ASCII colon character (':' = U+003a)
ASCII lower-case characters ('a'..'z' =
U+0061..U+007a)
ASCII digit characters ('0'..'9' = U+0030..U+0039)
Underscores are NOT allowed
No special characters, other than ASCII colons, dots and dashes, or white spaces
are allowed. The fully qualified name format enables storage administrators to assign
meaningful names to storage devices and manage devices more easily. The unique
identifier component can be a combination of department, application, manufacturer
name, serial number, asset number, and any tag useful for recognizing and
managing a storage resource.
UC434S F.00
8 -21
iSCSI name examples

iSCSI name examples
Windows
iSCSI name examples:
Linux
# cat /etc/initiatorname.iscsi
## DO NOT EDIT OR REMOVE THIS FILE!
## If you remove this file, the iSCSI daemon will not start.
## If you change the InitiatorName, existing access control lists
## may reject this initiator. The InitiatorName must be unique
## for each iSCSI initiator. Do NOT duplicate iSCSI
InitiatorNames.
## InitiatorName=iqn.1987-05.com.cisco:01.4f38fd6e357
InitiatorName=iqn.1987-05.com.cisco:01.rh3u5.Rack20-02
8 -22
UC434S F.00
iSCSI
iSNS
iSNS
An iSNS implementation provides four primary services:

Name Registration and Storage Resource Discovery
Discovery Domains and Login Control
State Change Notification
Bidirectional Mappings Between Fibre Channel and
iSCSI Devices
iSNS
Discovery using iSCSI names can be performed using the Internet Storage Name
Service (iSNS) or other resource locator. As implied by the structure of iSCSI names,
either a distributed or centralized DNS-type look up facilitates mapping of iSCSI
names required for iSCSI log in to actual iSCSI network addresses.
Name Registration and Storage Resource Discovery

iSNS implementations allow all entities in a storage network to register and query an
iSNS database. Both targets and initiators can register with the iSNS database, and
each entity can inquire about other initiators and targets. For example, a client
initiator can obtain information about target devices from an iSNS server.
Discovery Domains and Login Control

Administrators can use the Discovery Domains to divide storage nodes into
manageable, non-exclusive groups. By grouping storage nodes, administrators are
able to limit the login process of each host to the most appropriate subset of targets
registered with the iSNS, which allows the storage network to scale by reducing the
number of unnecessary logins and by limiting the amount of time each host spends
establishing login relationships.
Each target is able to use Login Control to delegate their access control and
authorization policies to an iSNS server. Such delegation is intended to promote
centralized management.
UC434S F.00
8 -23
State Change Notification

The State Change Notification (SCN) service allows an iSNS Server to issue
notifications about each event that affects storage nodes on the managed network.
Each iSNS client may register for notifications on behalf of its storage nodes, and
each client is expected to respond according to its own requirements and
implementation.
Bidirectional Mappings between Fibre Channel and iSCSI Devices

Because the iSNS database stores naming and discovery information about both
Fibre Channel and iSCSI devices, iSNS servers are able to store mappings of Fibre
Channel devices to proxy iSCSI device images on the IP network. These mappings
may also be made in the opposite direction, allowing iSNS servers to store
mappings from iSCSI devices to proxy WWNs.
8 -24
UC434S F.00
iSCSI
iSCSI target discovery

iSCSI target discovery
Discovery allows the iSCSI initiator to find (discover) targets to which it

has access.
Discovery Methods
iSCSI targets are configured on the initiator
Initiator would use a config file containing Target info
iSCSI initiator queries the Target
A SendTargets is issued to request a list of targets
Initiator uses Service Location Protocol (SLP)
To locate iSCSI targets or SNS without configuring addresses
Initiator queries a Storage Name Server (SNS, iSNS)
To locate iSCSI targets without configuring addresses
Discovery allows the iSCSI initiator to find (discover) targets to which it has access.
The following discovery methods are valid in an iSCSI environment:
iSCSI targets are configured on the initiator
iSCSI initiator queries the Target
To locate iSCSI targets or SNS without configuring addresses
Initiator queries a Storage Name Server (SNS, iSNS)
UC434S F.00
A SendTargets request is issued asking for a list of targets
Initiator uses Service Location Protocol (SLP)
Initiator would use a config file containing Target info
To locate iSCSI targets without configuring addresses
8 -25
iSCSI target discovery example

iSCSI target discovery example
Windows
iSCSI Discovery examples:
Linux
# cat /etc/iscsi.conf
.
.
.
# -------------------------# Discovery Address Category
# -------------------------#
DiscoveryAddress=33.33.66.66
# or
#DiscoveryAddress=10.4.100.1:3260
# or
#DiscoveryAddress=scsirouter1
#
8 -26
UC434S F.00
iSCSI
iSCSI operations
iSCSI operations
iSCSI login request to initiate
iSCSI session over TCP
iSCSI initiator
This persistent session now allows

authentication and the exchange of
certificates
iSCSI target
Multiple sessions may be active to

aggregate bandwidth and improve
performance
iSCSI initiator
iSCSI target
iSCSI is a connection-oriented command and response protocol. An iSCSI session

begins with an iSCSI initiator connecting to an iSCSI target (typically, using TCP) and
performing an iSCSI login. This login creates a persistent state between initiator and
target, which can include initiator and target authentication, session security
certificates, and session option parameters.
Once this login has been successfully completed, the iSCSI session continues in full
feature phase. The iSCSI initiator can issue SCSI commands encapsulated by the
iSCSI protocol over its TCP connection, which are executed by the iSCSI target. The
iSCSI target must return a status response for each command over the same TCP
connection, consisting of both the completion status of the actual SCSI target device
and its own iSCSI session status. An iSCSI session is terminated when its TCP session
is closed.
UC434S F.00
8 -27
iSCSI authentication
iSCSI authentication provides a mechanism to authenticate
iSCSI initiators requesting access to storage devices
(Targets)
Challenge Handshake Authentication Protocol (CHAP) is
one authentication method to pass user name and
password information between initiator and targets.
8 -28
iSCSI authentication provides a mechanism to authenticate iSCSI initiators

requesting access to storage devices (Targets)
Challenge Handshake Authentication Protocol (CHAP) is one authentication

method to pass user name and password information between initiator and
targets.
UC434S F.00
iSCSI
iSCSI CHAP examples

iSCSI CHAP examples
Windows
iSCSI CHAP examples:
Linux
# cat /etc/iscsi.conf
.
.
----------------------# AUTHENTICATION SETTINGS
# ----------------------# To globally configure a CHAP username and password for
initiator
# authentication by the target(s), uncomment the following
lines:
#
#OutgoingUsername=mpx100-user
#OutgoingPassword=igotasecret
#
# The maximum length for both the password and username
is 256 characters.
UC434S F.00
8 -29
IP Security
IP Security
Security
IKE
IPSEC
Authentication processes
Kerberos v.5
CHAP
Radius
SPKM -1 and SPKM -2
IP security
The maturity of IP based security makes SOIP attractive:
8 -30
IKE
IPSEC
Authentication processes
Kerberos v.5
CHAP
Radius
SPKM -1 and SPKM -2
UC434S F.00
iSCSI
iSCSI advantages & disadvantages

iSCSI advantages & disadvantages
Advantages
Disadvanatges
Shorter learning curve then Fibre

Channel for network and server
administrators
Higher overhead per packet than Fibre

Channel jumbo frames can be used to
alleviate this issue
High data availability built into the TCP

specification
Able to leverage other Ethernet and TCP

innovations and standards, such as
QoS, IPSec and IP Trunking
iSCSI exposes data to the network create

a dedicated, secured network for all block
level transmissions
Low cost compared to Fibre Channel, as

software initiators are available for most
operating systems
Consolidation of storage and
centralized backups and management
iSCSI increases network load and could

lead to congestion assign a dedicated
subnet for all iSCSI traffic
iSCSI protocol is not as widely tested as
SCSI or Fibre Channel
iSCSI places a higher load on the host CPU
use a dedicated iSCSI host adapters for
initiators
It has to be noted that iSCSI is an affordable means to integrate a lower performing

storage in to a 1Gbit/sec Ethernet providing shared storage for departmental use.
At 10 Gbit/sec, iSCSI loses much of its publicized cost advantage. By using a
10Gbit/ sec Ethernet implies that the applications being hosted require high
reliability and performance. At 1Gbit/sec standard NICs can be used, however
when implementing on 10Gbit/sec network server performance is enhanced by the
use of iSCSI cards which utilize auxiliary components like TOE (TCP off-load Engine)
or iSER (iSCSI Extensions for RDMA), which helps to avoid multiple memory copies of
SCSI data between the interface and application memory. These types of cards
(TOE and/or iSER) can add significant cost per attached server compared to an
8Gbit/sec FC HBA, and could undermine the value proposition of iSCSI at 1
Gbit/sec.
UC434S F.00
8 -31
P4000 HP StorageWorks iSCSI SAN
P4000 is HP StorageWorks SAN which is a low-cost but high-capability SAN based

on HP server hardware. All communications with the SAN, both SAN data and
management commands, are transferred over normal LAN connections.
The SAN generally includes a collection of StorageWorks SAN servers, called
Storage Nodes. Control software (SAN/iQ) runs on the Storage Nodes and handles
all communication and data management on the Storage Nodes. Administrators
access the SAN from a management system running the Centralized Management
Console (CMC) software.
8 -32
UC434S F.00
iSCSI
HP StorageWorks iSCSI SAN Recommended architecture
Unlike a fibre-channel SAN, all data transfers in an iSCSI SAN go over normal LAN
lines. Because of the heavy volume of data transferred in and out of Storage
Nodes, HP strongly recommends designing your network with isolated business LAN
and storage LAN segments.
In the diagram above, business traffic between user workstations and application
servers runs on a corporate LAN. All SAN traffic runs on a separate storage LAN.
The application servers connect to both LAN segments, making them accessible to
both users and the SAN.
Because the CMC must communicate directly with Storage Nodes, it cannot be
located only on the business LAN. There are two common configuration choices:
either dedicate a management system to CMC use, and connect the system directly
to the storage LAN (as shown above); or install the CMC software on one of the
application servers, and access it remotely from any system on the business LAN.
The first solution offers better security, and the second solution is more flexible and
convenient.
UC434S F.00
8 -33
Centralized Management Console (CMC)
The CMC is the primary interface for configuring and managing the Storage Nodes
in the SAN. This slide shows the areas of interest in the CMC interface.
The Launch Pad opens in the Content Pane when you run the Centralized
Management Console (CMC) for the first time. The Launch Pad offers several
Wizards to simplify the SAN setup process. For example, you can select the Find
Nodes Wizard in the Content Pane to locate the Storage Nodes available on your
network.
8 -34
UC434S F.00
iSCSI
CMC Navigation
The CMC displays different entities such as Management Groups, Storage Nodes,
Clusters, and Volumes in the Navigation Pane. Entities also have sub-entities or
attributes that allow you to configure the entities.
Simply expand the navigation tree by clicking on the + next to an entity this opens
the entity and, if appropriate, logs you in.
UC434S F.00
8 -35
Hierarchical Structure
The Navigation Pane displays the objects and configuration options you will use to
set up the SAN. In this slide you can see a Management Group (called MG1) that
contains several configuration options and a cluster (C1) with two Storage Nodes.
Select any of these objects to open and edit them.
8 -36
UC434S F.00
iSCSI
HP StorageWorks SAN Concepts
SAN/iQ is the control software running on the Storage Nodes. It controls all lowlevel data management such as disk striping, data replication across Storage
Nodes, and communication with the application servers. You do not normally
interact directly with SAN/iQ, but it controls all operations in the SAN.
CMC is the management interface you will use to communicate with SAN/iQ and
configure the SAN. When you first run CMC, you will tell CMC to find the
available Storage Nodes in your network. CMC will add them into an Available
Nodes pool.
You will then create Management Groups, which collect Storage Nodes into an
entity where they can be managed.
Within the Management Group you will create Clusters, which contain a subset of
the Storage Nodes in the Management Group. Clusters distribute data across all
Storage Nodes for increased performance and data protection.
You carve out Volumes (LUNs) from the space in a Cluster. Once you have
created your desired Volumes you can present them to remote application servers,
snapshot them (make point-in-time images available for later access), and do other
operations on them.
UC434S F.00
8 -37
Network RAID data mirroring
This slide illustrates one form of data distribution and protection used in HP
StorageWorks SAN: volume mirroring. This function is called Network RAID
because it operates very much like RAID in a disk controller, but using Storage
Nodes instead of individual disks.
The first example above uses Network RAID-0. In hardware RAID, RAID-0 stripes
data across multiple disks for higher performance. In the same way, Network RAID0 stripes data blocks across multiple storage nodes. Block B1 goes onto the first
Storage Node, block B2 goes onto the second node, and so on.
Hardware RAID-0 provides no protection against data loss, and the same is true for
Network RAID-0. You can instead use Network RAID-10, which provides varying
levels of data replication across the SAN. The second example above uses Network
RAID-10 with 2-way mirroring. Each block is written to two separate Storage Nodes,
so any single Storage Node can fail without loss of data. SAN/iQ supports 2-way
mirroring with no performance penalty, since it writes to two Storage Nodes
simultaneously.
SAN/iQ also supports 3-way mirroring (as shown in the third example) and 4-way
mirroring. 4-way mirroring is particularly useful for high-availability multi-site
installations. You can configure the SAN to have two copies of each data block at
each of two different geographical locations. Thus you could lose access to one of
the sites, and even lose one of the Storage Nodes at the remaining site, without
losing access to your data.
8 -38
UC434S F.00
iSCSI
SAN/iQ also supports network RAID with parity, called Network RAID-5 and
Network RAID-6. Network RAID-5 writes 4 blocks of data and 1 block of parity
across a minimum of 5 Storage Nodes, and can survive the loss of any single
Storage Node. Network RAID-6 writes 4 blocks of data and 2 copies of parity
blocks across a minimum of 6 Storage Nodes, and can survive the loss of any two
nodes.
UC434S F.00
8 -39
Configuring an HP StorageWorks SAN
The steps above will configure the SAN and present volumes to target application
servers.
These steps are explained in detail in the following slides.
8 -40
UC434S F.00
iSCSI
Discovering Storage Nodes
After launching the CMC, you must find the Storage Nodes in your network. CMC
has several methods to accomplish this.
UC434S F.00
8 -41
Creating a Management Group
When your CMC has found your Storage Nodes, you can collect them into
Management Groups. Management Groups have several functions that are beyond
the scope of this example. You just need to create a Management Group so you can
allocate some of your storage into Volumes.
8 -42
UC434S F.00
iSCSI
Creating a Cluster
Management Group is created by using a Wizard interface. This Wizard also steps
you through creating your first Cluster and Volume. The main operation in Cluster
creation is to choose which Storage Nodes are to be included in the Cluster.
UC434S F.00
8 -43
Creating a Volume
The Wizard next steps you into creating the first Volume. At this point the name and
size of the Volume, and the Cluster in which its storage resides is specified.
At this point you can also specify the data protection level (Network RAID) for the
Volume, and whether the Volume is Thin Provisioned or Fully Provisioned. A ThinProvisioned volume consumes only enough space in the Cluster to hold the data
currently in the Volume. The Volume grows as needed as new data is written to the
Volume, up to the maximum size specified at Volume creation time.
8 -44
UC434S F.00
iSCSI
Creating a Server
SAN/iQ uses objects called Servers to represent the connection between a Volume
and its target application server(s). After a volume is created, a server object is
created which is used to specify the application server that is allowed to connect to
it.
UC434S F.00
8 -45
Connecting a Volume to a Server
Now that you have created both the Server and the Volume, you can connect to
them by choosing either the Server or the Volume, that completes the SAN
configuration.
At this point you must go to the application server and configure the iSCSI Initiator to
point to the SAN Volume. Once the iSCSI Initiator connects to the Volume, the
Volume becomes visible to the server OS. Mount the volume, using the appropriate
process for your OS, and you are ready to access the SAN from your applications.
8 -46
UC434S F.00
iSCSI
Final Result
UC434S F.00
8 -47
Lab activity
Lab
activity
Module 8, Lab 1 - iSCSI LUN
Mapping
37
8 -48
uc434s c.01
2009 Hewlett-Packard
uc434sDevelopment
c.01 2009
Company,
Hewlett-Packard
L.P.
Development Company, L.P.
UC434S F.00
SAN extension
Module 9
Objectives
Objectives
Discuss SAN extension technologies and implementations
Describe FCIP and its role in SAN extension

FCIP performance
FCIP security
UC434S F.00
Explain Fibre Channel routing implementations in a SAN
9 -1
What is SAN extension?

What is SAN extension?
SAN Extension is an ISL connection between two Fibre

Channel switches over extended distances
Extended distances are considered to be

75m for 8Gb/s Fibre Channel ISLs
Any distance between a pair of Fibre Channel over IP
products
What is a SAN extension?

With the advent of extension technologies specifically developed for the transport of
data, you can consolidate, simplify, manage, and integrate storage in Fibre Channel
SAN fabrics within the enterprise to further exploit networking investments and lower
the cost to manage global storage.
A SAN extension is considered an inter-switch link (ISL) connection between two
Fibre Channel switches over extended distances. Extended distances are considered
to be:
Any distance between a pair of Fibre Channel over IP products
Whether it is called SAN Extension or SAN Bridging, HP seamlessly integrates these

new technologies into the benefits of HP Fibre Channel SANs.
9 -2
UC434S F.00
SAN extension
Why extend the SAN?

Why extend the SAN?
The growing need for storage data
Available bandwidth
Distance
IP technologies extend the leverage of installed SAN to
new constituents
Stranded remote servers
File-based storage applications (NAS/SAN fusion)

Uniting SAN islands
IP technologies extend control of the IT infrastructure

Shared tools
Integrated solutions
The growing need for storage data that is permeating the business community,
coupled with the available bandwidth afforded by IP networks or wave division
multiplexing (WDM), for example, are making SAN extension an increasingly
attractive option to grow the storage network. With SAN extension, users can
connect to data centers at opposite ends of a campus, metropolitan, and wide-area
environment. The challenge is to do so at full-wire speed, with the same reliability
and availability as the storage traffic within each data center.
IP technologies extend the leverage of installed SAN to new constituents for the
following:
Stranded remote servers
File-based storage applications (NAS/SAN fusion)
Uniting SAN islands
IP technologies extend control of the IT infrastructure by utilizing:
Shared tools
Integrated solutions
UC434S F.00
9 -3
HP Supported SAN extension technologies

HP Supported SAN extension
technologies
Fibre Channel long distance technologies

Long wave transceivers
DWDM
CWDM
IP data protocol technologies

FCIP
iFCP
iSCSI
HP Supported SAN extension technologies

HP supports the following technologies for Fibre Channel ISL SAN extension:
9 -4
Fibre Channel long distance technologies
Long-wave transceivers
Dense Wavelength division multiplexing (DWDM)
Coarse Wave Division Multiplexing (CWDM)
IP data protocol technologies
FCIP
iFCP
iSCSI
UC434S F.00
SAN extension
SAN extension distance summary

SAN extension distance summary
Increasing distance
Data
center
Optical
FC over Dark Fiber

FC over CWDM
FC over DWDM
FC over SONET/SDH
IP
ONS15454
Campus
200 km HP limit
Metro
Regional
250km
256 BB_Credits at
2Gb/s
National
Global
500km
256 BB_Credits at
1Gb/s
~100km
~320km
~500km
~1400km
(2G)
~2800km
~100km
(1G)
FCIP
~20,000km
(1G)
FCIP upper distance measure is approximately way around the globe.

The term dark fiber typically refers to fiber optic cabling that has been laid, but
remains unlit or unused. This is the simplest, but not necessarily the most cost effective
or scalable method for extending SANs over distance.
UC434S F.00
9 -5
Long Wave Transceivers

Long Wave Transceivers
Long Wavelength SFP 1550nm
2Gb/s 110km maximum distance
Application: GbE and 1 & 2Gb/s
Fibre Channel
Long Wavelength XFP 1550nm

10Gb/s 80km maximum distance
Application: SONET, 10G Ethernet,
10G Fibre Channel
Optical Small Form-factor Pluggable (SFP) transceivers are available in short- and
long-wavelength versions. The 4 Gb/s and 2 Gb/s transceivers are known as small
form-factor pluggables (SFPs) and use LC style connectors. The 1 Gb/s transceivers
can be LC SFPs or gigabit interface converters (GBICs), which use SC style
connectors
Short wavelength transceivers transmit at 850 nm and are used with 50 or 62.5 m
multimode fiber cabling. For fiber distances greater than several hundred meters
long-wavelength transceivers are used with 9 m single-mode fiber, and typically
operate in the 1310 or 1550 nm range.
Optical transceivers often provide monitoring capabilities that can be viewed through
FC switch management tools, allowing some level of diagnostics of the actual optical
transceiver itself.
The 8 Gbps sfp require a license this applies to the Brocade 300, 5100, and 5300
switches. Without this 8G license even if the correct 8G sfp is installed the maximum
speed the port would operate will be 4Gbps. If a license has been obtained and
installed on the switch, the commands portdisable and portenable on the individual
ports or a switchdisable and switchenable command to enable all the ports will have
to be performed to enable the 8 Gbps functionality on the ports.
9 -6
UC434S F.00
SAN extension
HP supports the following long-wave transceivers:
10km GBIC
100km GBIC
10km SFP
35km SFP
100km course wave division multiplexing (CWDM) SFP
Long-wave transceivers are supported on HP B-Series, HP C-Series, and HP M-Series

product lines. B-Series Fibre Channel switch products support 10km and 100km
GBICs (certain switch models), 10km and 35km SFPs, and 100km Coarse Wave
Division Multiplexing (CWDM) SFPs. The B-Series MP Router supports 10km and
35km SFPs. C-Series Fibre Channel switch products support 10km SFPs and 100km
CWDM SFPs. M-Series Fibre Channel switch products support 10km and 35km SFPs.
UC434S F.00
9 -7
Coarse Wave Division Multiplexing
CWDM is much less costly than DWDM because the channel spacing is only 20nm
and much less precise.
CWDM provides 8 channels between two CWDM Multiplexers over a single fiber
pair.
CWDM Multiplexers are usually un-powered devices containing a very accurate
prism to multiplex 8 separate wavelengths of light along a single fiber pair. Max
distance is approx 100Km.
HP offers a CWDM technology solution that involves concepts similar to Dense Wave
Division Multiplexing (DWDM) but is less expensive, less expandable (maximum
eight channels) and works over a distance of 100km. CWDM allows up to eight
1Gbps or 2Gbps channels (or colors) to share a single fiber pair. Each channel uses
a different color or wavelength transceiver. These channels are networked with a
variety of wavelength specific add-drop multiplexers to enable an assortment of ring
or point-to-point topologies.
Note: HP supports the use of all CWDM products as Fibre Channel ISLs provided the
CWDM equipment is configured to 1Gbps or 2Gbps data rates this can give
distances up to 100KM, or 4Gb/s to a distance of 40KM. Hp does not implement
time division multiplexing or any additional conversion method that alter the data
links other than multiplexing different wavelengths.
9 -8
UC434S F.00
SAN extension
Dense Wave Division Multiplexing
A single fiber pair connecting two FC switches together through an ISL provides a
single channel (wavelength of light) between the two switches.
DWDM enables up to 80 channels to share a single fiber pair by dividing the light
up into discrete wavelengths or lambdas separated by approx 1nm spacing around
the 1550nm wavelength.
Wavelength Division Multiplexing devices can be used to extend the distance
between two Fibre Channel switches. These devices are transparent to the switches
themselves and do not count as an additional hop. The only consideration that
should be made to accommodate these devices is to have enough buffer-to-buffer
Adding dense or coarse wavelength division multiplexing (DWDM/CWDM) to basic
Fibre Channel allows greater distances between sites than long-distance GBICs and
SFPs. The difference between WDM and basic fiber configurations is the addition of
a multiplex unit on both sides of the intersite link.
When using WDM, consider the following:
Always ensure WDM installation conforms to vendor specifications, and performance
is affected by distance and/or limited buffer-to-buffer credits on the Fibre Channel
switch. Switch vendors may limit the maximum distance between sites and apply
additional configuration rules for WDM configurations:
Connecting the switch to the WDM unit typically requires one switch-to-WDM
interface cable per wavelength of multimode fiber.
Note: Switches may require an Extended Fabric license.
UC434S F.00
9 -9
Time Division Multiplexing (TDM) takes multiple client-side data channels, such as
FC, and maps them onto a single higher-bit-rate channel for transmission on a single
wavelength. TDM is used in conjunction with a WDM solution provides additional
scalability and bandwidth utilization. However because TDM sometimes relies on
certain FC primitives to maintain synchronization, it may require enhanced
configuration when the extended fabrics are enabled. By default, Extended Fabrics
E_Ports use ARB primitives (specific to Virtual Channels) as fill words between frames.
The Majority of TDM devices require idles as fill words. Configuring a B-Series switch
to use R_RDY flow control will remedy this problem and enable interoperability. The
Remote Switch option is enabled by issuing the portcfgislmode = 1 command, or all
switch models that use fos 3.1 and above.
Note: A license is required for switch models that use 2.x firmware.
Note: HP Continuous Access products are not supported with WDM products that
implement active protocol handling.
Note: HP CWDM Multiplexer solutions are not supported on B-Series and M-Series
switches.
B-Series switch productsAll Brocade WDM-certified products, listed by Brocade
switch or router model number, are supported. The Brocade Data Center Ready
Compatibility Matrix can be viewed at the following address:
http://www.brocade.com/data-center-best-practices/resource-center/index.page
Note: HP supports the use of all WDM products as Fibre Channel ISLs provided the
WDM equipment is configured to 1Gbp/s data rates up to 500KM distance, or
250KM at 2Gb/s or 4Gb/s to distance of 100KM and 40Km at 8Gb/s.
Hp does not implement time division multiplexing or any additional conversion
method that alters the data links other than multiplexing different wavelengths.
9 -10
UC434S F.00
SAN extension
FCIP
FCIP
Encapsulates fibre channel frames into IP packets

Tunnels packets through existing IP network
IP tunnels acts as a dedicated link
FCIP gateways perform fibre channel encapsulation into IP and reverse
B-series and M-Series fibre channel switches connect to FCIP gateways
through E-port
C-Series switches use plug-in modules for FCIP functionality
Transparent both to IP and fibre channel networks
Extends a SAN and creates a single fibre channel network
The Fibre Channel over IP (FCIP) protocol connects switches over a GbE-based
network.
FCIP is a protocol that encapsulates Fibre Channel frames into IP packets and tunnels
them through an existing IP network infrastructure to transparently connect two or
more SAN fabrics together. The IP tunnel acts as a dedicated link to transmit the
Fibre Channel data stream over the IP network, while maintaining full compatibility
with the Fibre Channel SAN.
FCIP gateways perform Fibre Channel encapsulation process into IP packets and
reverse that process at the other end.
Fibre Channel switches (B-Series and M-Series) connect to the FCIP gateways through
an E_Port for SAN fabric extension to remote locations. C-Series switches use plug-in
modules for FCIP functionality.
A tunnel connection is set up through the existing IP network routers and switches
across LAN/WAN/MAN.
This example shows a configuration that connects Fibre Channel SANs using an
Internet Protocol (IP) intersite link.
Note: The gateways at either end of the link must be from the same gateway family
to insure interoperability. Refer to the Continuous Access Data Replication Manager
SAN Extension Reference Guide for details.
UC434S F.00
9 -11
FCIP Protocol Mapping

FCIP Protocol Mapping
FCIP
Tunnel
FCIP
FCIP
FC-2
TCP
TCP
FC-2
FC-1
IP
IP
FC-1
FC-0
Link
Link
FC-0
Physical
Physical
IP Network
IP Network
Fibre Channel
Fibre Channel
Fibre Channel frames are transported across an IP network, encapsulated within a

FCIP packet through a FCIP tunnel. The Fibre Channel frames that are destined for
delivery to a device on a remote San are encapsulated into a TCP/IP packet, where
upon a standard IP header will be added.. This packet will then be delivered to the
next hop in the chain, usually a network router.
9 -12
UC434S F.00
SAN extension
Fibre Channel over IP

Encapsulation
FC frame
FCIP frame
Eth
IP
TCP FCIP
SOF
FC Hdr
FC payload
CRC EOF
SOF
FC Hdr
FC payload
CRC EOF
Before a FC Frame can be sent out via FCIP over a Gigabit Ethernet link, the
transmitting FCIP port encapsulates the FC frame in the payload of each of the four
protocols in the stack, FCIP, TCP, IP and Ethernet. The receiving FCIP port then strips
the Ethernet, IP, TCIP and then the FCIP headers, and if necessary reassembles the
FC frame if an fragmentation has occurred during transit and then forwards the FC
frame in to the fabric and onward to its destination.
UC434S F.00
9 -13

Before an FCIP tunnel can be created between two FC/FCIP Entities each will require the
following configuration:
1. A Static Ip address for each FCIP_LEP ( FCIP_Link End Point)
2. Two fixed TCP ports to send and receive data transmissions.
3. A WWN of the other end of the FCIP Link
Note: TCP port 3225 is used for FCIP Class of Service F
and TCP port 3226 is used for Class of Serves 2 and 3
FC Entity
FCIP_LEP
IP wan
IP wan
Multiple
FCIP_LEPs
(Link End Ports)
Tunnels
1-8
Multiple
FCIP_LEPs
(Link End Ports)
Tunnels
1-8
FCIP_LEP
FC Frame
FC Entity
FCIP_LEP
FCIP_LEP
FC Frame
FC Virtual Channels and FCIP tunnels are similar in concept, but the main difference
is that FCIP require an Ip addresses, a TCP port, and its parameters plus QoS
information, and the expected WWN of the device at the other end of the link. With
this information the FC Entity combines with the FCIP entity components to form an
interface between a FC fabric and an IP network
A FC entity contains FC specific components like Field Programmable Gate Arrays
(FPGAs), which is used to determine if Compression/decompression is required on a
packet, if it is the frame is forwarded to the relevant circuitry. The Field
Programmable Gate Arrays (FPGAs), is also used to handles TxID translation which
ensures that the Ip packet is delivered to the correct TCP connection on the correct
GbE port.
The protocol exchanges on the IP network are the responsibility of the FCIP entity,
which contains the FCIP control components, FCIP Link End Ports (FCIP_LEPs) and a
FCIP Data Engine (FCIP_DE). The control components are responsible for FCIP
protocol exchanges on the network, FCIP_LEPs are used to connect one end point of
a TCP connection to the other TCP FCIP_LEPs at the other side, and finally the
FCIP_DE handles FC frame encapsulation, de-encapsulation and transmission. Once
the tunnel is created and the FC frame is in the network all normal IP network routing
procedures apply.
By default, the FCIP feature creates two TCP connections for each FCIP link.
9 -14
One connection is used for data frames.

UC434S F.00
SAN extension
The other connection is used only for Fibre Channel control frames, that is,
switch-to-switch protocol frames (all Class F). This arrangement provides low
latency for all control frames.
Note: TCP port 3225 is used for FCIP Class of Service F and TCP port 3226 is used
for Class of Serves 2 and 3
UC434S F.00
9 -15
FCIP performance
Effective bandwidth is the amount of available bandwidth that can actually be

used, which also takes in to consideration, congestion, re-transmission and
dropped packets.
Latency, also referred to as delay is the measurement of time it takes a packet of
data travel from one point to another. Considerations include, distance, router
hop count, processing and packet size; compare this against FC delays which
are on 5 nanoseconds per meter.
Packet loss and congestion are synonymous in that congestion is the prime cause of
packet loss in a network. Therefore any packets lost would lead to a need for retransmission of the lost packet. Packet loss significantly degrades FCIP performance;
this is due to the fact the loss must be acknowledged from the receiving port before a
re-transmission can take place.
9 -16
One method of managing WAN bandwidth is by the use of FCIP Trunking,

which provides redundant paths protecting the WAN against transmission
failures. When an FCIP tunnel is formed logical circuits are created. Trunking is
able to combine these circuits that have been made between a pair of IP
addresses that are associated between the source and destination endpoints of
the FCIP tunnel.
Each FCIP circuit is assigned a metric, which is used in managing load leveling
and failover for FC traffic. FCIP Trunking uses the metric to determine if a circuit
is to be used for load leveling or failover.
UC434S F.00
SAN extension
Adaptive Rate Limiting (ARL) is performed on FCIP tunnel connections to change

the rate in which the FCIP tunnel transmits data through the TCP connections.
ARL uses information from the TCP connections to determine and adjust the rate
limit for the FCIP tunnel dynamically. This allows FCIP connections to utilize the
maximum available bandwidth while providing a minimum bandwidth
guarantee.
ARL applies a minimum and maximum traffic rate, and allows the traffic
demand and WAN connection quality to dynamically determine the rate. As
traffic increases, the rate grows towards the maximum rate, and if traffic
subsides, the rate reduces towards the minimum. If traffic is flowing error-free
over the WAN, the rate grows towards the maximum rate. If TCP reports an
increase in retransmissions, the rate reduces towards the minimum
UC434S F.00
9 -17
Network speeds
In general, the FCIP equipment supports Ethernet throughput of 10/100MB/s, 1Gbps

(Gigabit Ethernet) and 10GbE. Select the network connection to match the amount
of data to be transferred.
The speed of light through fiber is approximately 200,000km per second or 5
microseconds to travel 1km.
Network distance considerations
Some disk array controller use SCSI protocol to manage their storage devices. Before
a SCSI I/O can be transmitted, it must be encapsulated into Fibre Channel frames.
Because of SCSI protocol, a minimum of four trips over the long-distance link is
required.
These trips conceptually:
9 -18
Tell the remote site you want to transmit data
Wait for the acknowledgment from the remote site
Send the data to the remote site
Wait for the acknowledgment from the remote site
UC434S F.00
SAN extension
When sending data over fiber, the one-way transmission time is approximately 5
microseconds per km of optical cable. Because a minimum of four trips is required
for each SCSI data transfer, this translates to a total transmission delay per command
of 20 microseconds per km, or about 32.2 microseconds per mile. For example, if a
remote site is located 150 miles away from the local site, the total time will be 4,830
microseconds (4.83 milliseconds) for every data transfer. Because a typical I/O
operation on a non-Data Replication Manager (DRM) configuration with write-back
cache takes approximately 500 microseconds, long distances can have a significant
effect on performance.
Note: The preceding calculations for a link of 150 miles do not include any latency
induced by the FC-to-IP conversions or latency of the routers and switches in the
network.
Additional I/Os, either from additional LUNs on the same controller or from a
different controller, will require additional bandwidth. Care must be taken to
understand this principle.
Adding bandwidth to a given link at a given distance will not increase the time it
takes to complete an I/O operation. It will, however, allow you to add additional
I/Os from different LUNs, thereby consuming the available bandwidth.
Conversely, if enough bandwidth is not given to a link, then the number of I/Os per
second will decrease, possibly to the point of failure.
Note: The time it takes an I/O to complete an operation is more complex than this
example, and there are additional factors involved with this calculation. This
discussion is an attempt to help you understand the importance that distance latency
has on the time it takes to complete an I/O operation.
Network distance and latency examples

1.
1.0MB Link
Link Bandwidth: 1.0MB/s
Write size: 8KB
Available bandwidth divided by size of I/O equals maximum I/Os per second:
1.0MB/s = 125 I/Os per second
8KB per I/O
2.
50 Miles of Latency
Distance: 50 miles (80km)
Latency: 8 s/mile (5 s/km)
Write size: 8KB
Latency for 1 I/O per mile: 4 trips * 8 s/mile = 32 s per mile
UC434S F.00
9 -19
Latency for 1 I/O at 50 miles: 50 miles * 32 s/mile = 1.6ms per I/O

Reciprocal of total latency indicates maximum I/Os:
1.0 = 625 I/Os per second
1.6ms per I/O
I/Os multiplied by size of I/O = bandwidth used:
625 I/O per second * 8KB per I/O = 5MB/s
3.
150 Miles of Latency

Distance: 150 miles (241km)
Latency: 8 s/mile (5 s/km)
Write size: 8KB
Latency for 1 I/O per mile: 4 trips * 8 s/mile = 32 s per mile
Latency for 1 I/O at 150 miles: 150 miles * 32 s/mile = 4.8ms per I/O
Reciprocal of total latency indicates maximum I/Os:
1.0 = 208 I/Os per second
4.8ms per I/O
I/Os multiplied by size of I/O = bandwidth used:
208 I/O per second * 8KB per I/O = 1.6MB/s
In summary, when an IP network is used in a situation for which the local and remote
sites are located many miles apart, the speed of light through fiber can cause
unacceptable delays in the completion of an I/O transaction. Increasing the amount
of available bandwidth cannot solve this problem. Give careful consideration to these
factors when matching your needs and wants to a particular application.
9 -20
UC434S F.00
SAN extension
Basic SCIS WRITE OPERATION
The basic SCSI read operations are simple: in that the SCSI initiator initiates a
request for data and the SCSI target responds with all of the requested data without
further acknowledgements or round-trips across the connection. When there is a
SCSI write operation it is more involved, because it requires two round-trips between
the SCSI initiator and target. Before an initiator is allowed to send any data to the
target, the initiator must first notify the target of the impending write to find out if
space is available in the receiving buffer. The target then responds with a transfer
ready message if space is available. When a SCSI initiator intends to write an
Information Unit (IU) it sends a message, which is part of an exchange called a
sequence, to the SCSI target indicating the size of the write. The SCSI target then
responds with a transfer ready (FCP_XFER_RDY) sequence specifying how much
data the initiator is allowed to transfer, which is usually the size of the entire write.
After all the data has been transferred, the target sends the command completion
sequence (FCP_RSP) back to the initiator, acknowledging that it has received and
stored all of the information written by the initiator. Upon receipt of that response,
the write is complete.
When long distances and significant latencies exist between target and initiator,
SCSI write operations can involve multiple handshake messages between target and
initiator to transfer a SCSI write data sequence (FCP_DATA) from the initiator to the
target.
When the SCSI write operation is performed over distance, each additional roundtrip communication between the SCSI application client and device server increases
the time needed to complete the overall write operation. This results in degraded
UC434S F.00
9 -21
application performance. Unless measures are taken to mitigate the negative impact
of latency, storage data transfers over a few kilometres in distance will suffer to a
degree considered unacceptable for a large number of storage applications.
9 -22
UC434S F.00
SAN extension
Brocade Fastwrite
The Brocade FastWrite feature is designed to overcome the latency effects of SCSI
write operations, this is achieved without compromising data integrity and security.
FastWrite allows the entire data sequence of the SCSI operation to be transported
across the link, without the inefficiencies of waiting for the transfer ready command
(FCP_XFER_RDY) to travel back across the high-latency environment
When a write operation is detected, the Brocade 7500 SAN Router forwards the
write command to the target in the standard way. Commands are therefore delivered
to the target in the same order that they were issued by the initiator. However, the
Brocade 7500, acting as a virtual target, immediately issues an FCP_XFER_RDY
sequence to the initiator, prompting it to transmit the entire data sequence
(FCP_DATA) for the write operation. The Brocade 7500 transfers the data across the
high-latency environment to the remote SAN router. The remote target device then
interacts with the remote SAN Router, which acts as a virtual initiator. FCP_XFER_RDY
issued by the target is handled directly by the remote SAN Router, as if the router
were the real initiator issuing the data for the write operation. FastWrite allows the
SAN Routers to expedite transfer of the SCSI write data sequence, without having to
wait for potentially numerous round-trip handshake messages to travel back and forth
between target and initiator.
There are two options available for enhancing open systems SCSI tape write I/O
performance.
FCIP fastwrite and tape pipelining.
FC fastwrite.
UC434S F.00
9 -23
FCIP fastwrite and tape pipelining are implemented together. FC fastwrite is a FC-FC
routing alternative that disables the local Ethernet ports (ge0 and ge1); this makes it
impossible to configure FCIP fastwrite and tape pipelining and FC fastwrite on the
same 7500 or FC4-18i blade.
To configure a fcip tunnel use the following commands:
portcfg fciptunnel
portshow fciptunnel
Note: Brocade FastWrite is available with either FC-based or FCIP extension.
Note: FCIP FastWrite is supported with XP Continuous Access but is not supported
with Continuous Access EVA.
9 -24
UC434S F.00
SAN extension
Brocade FCIP Fastwrite + Tape pipelining
Open Systems Tape Pipelining (OSTP) can be used to enhance open systems SCSI
tape write I/O performance, it builds upon the fastwrite feature to optimize
sequential I/Os to a remote device. When the FCIP link is the slowest part of the
network, OSTP can provide accelerated speeds for read and write I/O over FCIP
tunnels. To use OSTP, you need to enable FCIP Fastwrite and Tape Pipelining.
FCIP Fastwrite accelerates the SCSI write I/Os over FCIP.
Tape Pipelining accelerates SCSI read and write I/Os to sequential devices (such
as tape drives) over FCIP, which reduces the number of round-trip times needed to
complete the I/O over the IP network and speeds up the process. Each GbE port
supports up to 2048 simultaneous accelerated exchanges.
Both sides of an FCIP tunnel must have matching configurations for these features to
work. FCIP Fastwrite and Tape Pipelining are enabled by turning them on during the
tunnel configuration process. They are enabled on a per-FCIP tunnel basis.
The tape pipelining process: Firstly the fastwrite operation initiates the local gateway
to send a XFR_RDY command after each SCSI write command is received, resulting
in only one round trip per write operation. Once the data is sent by the initiator to
the local gateway FCP_DATA the local FCIP port immeadiately responds witha
FCP_RSP mesaage. The initiator then interprets this message as the completion of the
write and begins the process of sending the next write. Because FastWrite is
enabled, the local FCIP port is buffering data, allowing it to keep the pipe full and
maintain a steady flow of data to the remote tape device.
UC434S F.00
9 -25
Tape pipelining, require the request and corresponding response traffic to traverse
the same VE_Port tunnel across the metaSAN. To ensure that the request and
response traverse the same VE_Port tunnel, you must set up Traffic Isolation zones in
the edge and backbone fabrics.
Set up a TI zone in an edge fabric to guarantee that traffic from a specific
device in that edge fabric is routed through a particular EX_Port or VEX_Port.
Set up a TI zone in the backbone fabric to guarantee that traffic between two
devices in different fabrics is routed through a particular ISL (VE_Ports or E_Ports)
in the backbone.
This combination of TI zones in the backbone and edge fabrics ensures that the
traffic between devices in different fabrics traverses the same VE_Port tunnel in a
backbone fabric.
9 -26
UC434S F.00
SAN extension
Cisco Write Acceleration

Cisco Write Acceleration
Write Acceleration spoofs
XFER_READY
Can spoof up to 32MB of
outstanding I/Os
Single round trip over
WAN
Enables 2x distance at same
latency
For some apps, can
achieve >2x throughput
FCIP with Write Acceleration

After the initiator issues a SCSI FCP Write, an FCP_XFER_RDY is immediately
returned to the initiator by the MDS 9000.
The initiator can now immediately send data to its target across the FCIP Tunnel. The
data is received by the remote MDS and buffered.
At the remote end, the target, which has no knowledge of Write Acceleration,
responds with an FCP_XFER_RDY. The MDS does not allow this to pass back across
the WAN.
When the remote MDS receives FCP_XFER_RDY it allows the data to flow to the
target.
Finally when all data has been received, the target issues a FCP_RSP response or
status, acknowledging the end of the operation (FC Exchange) Write Acceleration
will increase write I/O throughput and reduce I/O response time in most situations,
particularly as the FCIP RTT increases.
To view configuration of FCIP including write acceleration use the:
show interface fcip command.
UC434S F.00
9 -27
The write acceleration feature is disabled by default and must be enabled on both
sides of the FCIP link.
If it is only enabled on one side of the FCIP tunnel the write acceleration feature will
be turned operationally off.
9 -28
UC434S F.00
SAN extension
FCIP Compression
FCIP Compression
Improvements
compression
to data transfer rate due to
LZS hardware compression good

Deflate hardware compression better
Compression is used as a mechanism to increase overall throughput on slow speed

WAN links. The achievable compression ratio depends on the nature of the data.
The use of data compression allows users to achieve two major objectives. The first is
the ability to reduce the amount of overall traffic on a particular WAN link. This is
achieved when a data rate equal to the WAN link speed is compressed, thus
reducing the total amount of data on the WAN link and allowing the WAN link to
be used by other IP traffic.
Some hardware uses the IPPCP/LZS (RFC2395) lossless compression algorithm for
compressing data. The IPPCP/LZS compression compresses only the TCP headers
and payload of the FCIP frame as shown here. This allows the resulting compressed
IP frame to routed through an IP network and still be subject to Access Control Lists
(ACLs) and QoS mechanisms based on IP address and TCP port numbers.
The type of the data in the data stream determines the overall achievable
compression ratio for a given compression method. Typical data mixes should
achieve around 2:1 compression.
Testing compression with data comprised of all 0x0s or 0xFs or other repeating
patterns will artificially increase the resultant compression ratio and will probably not
be representative of the compression ratio that you can achieve with real user data.
In order to better compare compression methods, use either an industry standardized
test file or a test file that is representative of the real data that will be sent through the
FCIP tunnel.
UC434S F.00
9 -29
IP network considerations
IP network considerations
Do I use my existing WAN link or provision a separate one?
Depends on:
The type of storage I/O
Use of the existing network
Storage I/O type
Use existing IP network?
Factors
Mirrored I/O or continuous I/O

throughput over the FCIP intersite link.
A separate network is
recommended.
For peak performance for your

current network, and for peak
Storage I/O performance, a separate
network is recommended.
Data migration or ad hoc data

updates.
The use of your existing network

may be possible.
Data migration is a one-time

movement of data for upgrade or
maintenance purposes.
Ad hoc data updates is more of a
burst of data from one site to
another for remote backups,
database content delivery, etc.
It is possible to use your extisting
network; however, the network
performance may be significantly
affected.
The ability to use your existing network with FCIP depends on the type of storage
I/O you plan to do and the traffic already existing on your current network. The key
consideration is whether you have enough unused or available bandwidth from your
network to continue the current network load, accommodate future growth, and
handle FCIP SAN load demands.
9 -30
UC434S F.00
SAN extension
IP network best practices

To minimize bandwidth appropriation

Create VPNs with QoS through premise routers for the FCIP circuit
Create separate physical networks
Guarantee the bandwidth using a third-party router/QoS vendor
As distance has a dramatic effect on the amount of work that can be

completed across a link, site planning should include
Using the shortest possible distance between remote sites
Minimizing the amount data transferred over the FCIP link
Designing a plan to add additional storage I/O that will not impact
normal data traffic
Consider additional controller pairs to effectively use available
bandwidth

Most IP networks do not manage bandwidth to each individual connection. As traffic
increases because of other demands on the network, bandwidth can be robbed from
the FCIP intersite link.
You can use the following techniques to minimize this effect:
Create virtual private networks (VPNs) with Quality of Service (QoS) through
premise routers for the FCIP circuit.
Create separate physical and dedicated networks.
Guarantee the bandwidth, latency, and latency jitter using a third-party
router/QoS vendor.
As mentioned, distance has a dramatic effect on the amount of work that can be
done across a link. Therefore, site planning should include:
Using the shortest possible distance between remote sites
Minimizing the amount data transferred over the FCIP link
UC434S F.00
Designing a plan to add additional storage I/O that will not impact normal
data traffic
Considering additional controller pairs to effectively use available bandwidth.
9 -31
FCIP Security
FCIP Security
DWDN, CWDM and SONET are considered secure
FCIP over public IP pose serious security risks

A 3rd party encryption solution could be used which would increase latency
(delay), or a Brocade security Encryption switch which performs the encryption
within the San. Encryption is also available on Brocade blade an Fs8Encryption blade, Cisco Storage Media Encryption (SME)
Optical DWDM, CWDM, or SONET/SDH links are considered relatively secure due
to the inherent difficulty of tapping into optical fiber. However, security on FCIP
tunnels that are routed over public IP is a serious issue. For regulated institutions like
financial companies, health care, and schools, encryption of data transmitted over
public networks is not just a good idea, it is a requirement.
FCIP gateway products on the market today do not provide integrated encryption.
Users must rely on routers or VPN appliances at the WAN edge to encrypt storage
traffic. Not only does this still leave storage traffic vulnerable to interception up to the
WAN edge, but it may require users to buy yet more equipment if the existing routers
or VPN appliances cant support gigabit-speed storage traffic in addition to existing
WAN traffic loads.
9 -32
UC434S F.00
SAN extension
FCIP encryption
FCIP encryption
Standards-based IPSec protocol to secure FCIP and iSCSI
traffic
Hardware-based end-to-end authentication and encryption
B-Series
Supported on:
HP StorageWorks 400 MultiProtocol Router , MP Router

Blade (FR4-18i)
C-Series
Encryption Data at rest

Cisco Storage Media Encryption (SME)
Brocade Encryption Switch
Brocade Encryption FS8-18 Blade
Supported on:
MDS 9216i, MPS 14+2

module
C-Series and B-Series switches support Traffic encryptionIPSec over FCIP. At this
moment in time this can only be created on tunnels using IPv4 addressing.
To verify the IPsec information on C-Series switch use the show set of commands
show ip access-list usage
show ip access-list
show crypto transform-set domain ipsec
show crypto map domain ipsec
In a B-Series environment IPSec policies are managed using the policy command.
policy --create type number [-enc encryption_method][-auth
authentication_algorithm] [-pfs off|on] [-dh DH_group] [-seclife secs]
policy --show ipsec 1
policy --show ike all
Viewing IPSec information for an FCIP tunnel in a B-Series environment portshow
fciptunnel <Slot/ge-port> <FCIP tunnel number> -ipsec
eg: portshow fciptunnel 8/ge0 3 ipsec.
Hardware encryption services are now available on the DCX Brocade switches by
the use of a FS8 18 blade, or by the use of a Brocade Encryption Switch, these
UC434S F.00
9 -33
products can provide up to 48 to 96 Gbit/sec encryption, this allows for the

encryption to be performed with in the san, before transmission on to the FCIP
network.
9 -34
UC434S F.00
SAN extension
FCIP advantages
FCIP advantages
Advantages:
Low-cost connectivity solution

Ubiquitous connectivity (IP)
No fixed distance limitation
Not reliant of Fibre Channel buffer credits
Integrates easily into existing network management scheme
Granular scalability by upgrading underlying transport
Disadvantages:
Higher latency than CWDM / DWDM

Fully merged fabric will segment if WAN connection fails
Need to reserve bandwidth across shared IP network (QoS)
Many proprietary product options based upon a standard
Advantages:
Low-cost connectivity solution
Ubiquitous connectivity (IP)
No fixed distance limitation
Not reliant of Fibre Channel buffer credits
Integrates easily into existing network management scheme
Granular scalability by upgrading underlying transport
Disadvantages:
Higher latency than CWDM / DWDM
Fully merged fabric will segment if WAN connection fails
Need to reserve bandwidth across shared IP network (QoS)
Many proprietary product options based upon a standard
UC434S F.00
9 -35
FCIP hardware
FCIP hardware
B-Series
C-Series
Fibre Channel (E, F, FL, EX) and Gigabit

Ethernet
18 ports (16 FC and 2 Ethernet)

Simple Name Server, Registered State
Change Notification (RSCN) and iSCSI

Gateway Service
Fabric services include the FC-FC

Routing
MDS 9216i and Multi-protocol Services module - 14 to 62

Fibre Channel ports plus two Ethernet ports,or 14 Fibre
Channel ports and up to ten 1Gb Ethernet or four 10Gb
Ethernet ports
8-port IP Storage Services Module
8 GbE ports
Supports FCIP and iSCSI
Service, FCIP Tunneling Service,

Advanced Zoning and WebTools
HP StorageWorks IP Distance Gateway
9 -36
Provides a low-cost Fibre-Channel to IP Gateway

FCIP protocol
Uses software compression
Has rate limiting to match GbE port to LAN, MAN, and WAN
Supports HP StorageWorks Continuous Access EVA (CA EVA)
Supports B-Series and C-Series switches
UC434S F.00
SAN extension
Fibre Channel routing overview
Fibre Channel routing overview

A single fabric can be divided into several subnetworks or logical groups of switches
or switch ports called virtual SANs (VSANs) or Virtual Fabrics. A group of VSANs or
Virtual Fabrics are still a SAN.
Fibre Channel routing facilitates the development and management of highercapacity SANs, significantly increasing device connectivity while maintaining Fabric
independence and fault domain isolation. With Fibre Channel routing, multiple
existing fabrics including Virtual Fabrics and VSANs are interconnected rather than
merged, overcoming individual fabric scaling limits. By enabling communication
between two or more independent SANs, routing increases levels of SAN scalability.
Note: In the context of Fibre Channel routing, the terms fabric and SAN are used
interchangeably.
Routing enables independent fabrics, Virtual Fabrics with the use of a back bone
switch or VSANs with inter-VSAN routing (IVR) to share devices dynamically without
the need to reconfigure or re-cable physical connections. Routed Virtual fabrics or
VSANs with IVR can consolidate management interfaces. Instead of one
management interface per fabric, there can be one per SAN, or two per SAN, if
redundant fabrics are used.
Note: HP does not support using the B-Series MP Router and C-Series IVR
functionality in the same SAN configuration.
UC434S F.00
9 -37
Additional Fibre Channel routing features include:
9 -38
Improved device access and sharing
Ability to share devices dynamically across multiple fabrics, Virtual Fabrics

or VSANs
Increasing device utilization
Centralized SAN fabric management
Common fabric management
Tape backup consolidation
UC434S F.00
SAN extension
Fabric and VSAN independence

Fibre Channel routing identifies data frames in a fabric or VSAN for

transfer to other fabrics or VSANs with IVR
Fabric services
Coordinates communication between switches
Manages device names, addresses, timestamps, utilities
WWN
Assigns devices to zones and defines devices for export
Meta SANs
Created when fabrics or VSANs are connected
Includes the physical fabrics, router, and LSANs
Import and export
Routing exports devices from one fabric or VSAN to another
Routing table
Reads the address information in each frame that it receives
Determines the destination and destination address

Fibre Channel routing identifies data frames in a fabric or VSAN for transfer to other
fabrics or VSANs with IVR. Because only data addressed to a device in another
fabric or VSAN passes through the router or routing function, a disruption of fabric
services in one routed fabric or VSAN is unlikely to propagate to another.
Fabric services
Fabric services coordinate communication between switches in a fabric or VSAN.
The fabric services manage device names and addresses, timestamps, and switch
utilities.
Routing connects devices in multiple fabrics or VSANs without extending fabric
services from one routed fabric to another. Each fabric or VSAN maintains a unique
fabric services configuration. Devices in a routed network can communicate across
logical SANs (LSANs) or VSANs despite having different fabric services
configurations. An LSAN is similar to a Fibre Channel zone, but can extend through
a router to include devices in other fabrics.
UC434S F.00
9 -39
World-Wide Name
A recognized naming authority assigns each Fibre Channel device a World-Wide
Name (WWN), which is a unique identifier. Use the device WWNs to:
Assign devices to zones
Define devices to export from one fabric or VSAN to another
Meta SANs
Routing creates a Meta SAN or extended VSAN when it connects fabrics or VSANs.
A Meta SAN is a configuration that includes the physical fabrics (subnetworks),
router, and LSANs. When forming a Mata SAN, you determine which fabrics require
connectivity and then specify the devices allowed to communicate across fabrics. The
router does not provide 100% any-to-any connectivity between fabrics, but it does
meet most SAN requirements.
Import and export

Routing also exports devices from one fabric or VSAN to another. An exported
device has an imported address in every destination fabric or VSAN to which it has
been exported. The address of the exported device in the source fabric or VSAN is
its exported address.
An imported device is a device as seen in a fabric when using its imported address.
An exported device is a device as seen in the fabric when using its exported
address.
Routing table
The routing function reads the fabric address information in each frame that it
receives, and then uses a routing table to determine the destination fabric or
destination VSAN and the address within that fabric or VSAN. The routing function
then transmits the frame to the address in the destination fabric.
9 -40
UC434S F.00
SAN extension
SAN scaling
SAN scaling
There are two methods for increasing the size of SANs:
Increase the Fibre Channel switch capability within a fabric

Connect independent fabrics using a Fibre Channel router, Virtual Backbone or
VSANs with IVR
Switch scaling limits

The switches that make up fabrics define the fabric limits.
Adding ports to a fabric means increasing the number of switches in the fabric or
increasing the number of ports per switch. For large fabrics, adding ports might not
be possible unless the limits for total port count and total switch count are increased.
Each Fibre Channel switch product line has its own limits for total port count and
switch count. You must ensure that a new or modified SAN design complies with
these limits.
Note: Other limits, such as hop counts and link distances, also apply.
UC434S F.00
9 -41
For a SAN design to meet the total port count and total switch count limits, the
following configuration restrictions are enforced:
The fabric size limit for total port or total switch count must not be exceeded.
The use of several small switches to reach a high total port count number is not
acceptable if the design exceeds the total switch count limit.
The use of several high-port-count switches is not acceptable if the design
exceeds the total port count limit.
For large configurations, HP defines the maximum supported port and switch counts.
9 -42
UC434S F.00
SAN extension
Fabric services limits

Increasing fabric size increases the overhead associated
with coordination
SNS provides a mapping between device names and their
addresses in a fabric
To ensure that the mapping is current, every switch in the
fabric implements SNS
Coordinating fabric services
When two fabrics are connected, their two sets of fabric
services merge to form a single set
Without routing, fabric scaling is limited by the ability of
the smallest fabric switch to participate in the distributed
fabric services system

Fabric services provide coordination between all switches in a fabric. Increasing
fabric size increases the overhead associated with coordination. Fabric services
include:
Fabric Login Server
State Change
Notification Server
Name/Directory Server
Zone Server
Key Server
Time Server
Simple Name Service
Simple Name Service (SNS) provides a mapping between device names and their
addresses in a fabric. To ensure that the mapping is current, every switch in the fabric
implements SNS.
UC434S F.00
9 -43
Coordinating fabric services

Each fabric maintains a unique set of fabric services. When two fabrics are
connected, their two sets of services merge to form a single set.
As fabrics grow, coordinating the fabric services across switches, hosts, and storage
devices becomes more challenging. It is difficult to match the fabric service
requirements for very small, inexpensive switches with those for large, high-end
switches. Without routing, fabric scaling is limited by the ability of the smallest fabric
switch to participate in the distributed fabric services system.
9 -44
UC434S F.00
SAN extension
Scaling by Routing
Scaling by Routing
Demand for higher port counts and connectivity between
devices in different fabrics or VSANs requires Fibre
Channel routing.
Routing improves scaling by connecting independent
fabrics or VSANs
Connectivity between fabrics or VSANs allows sharing of

resources, reducing unnecessary redundancy
Route between fabrics without affecting the total switch and

port count limits
A routed network is not the same as a single large fabric
or VSAN
Scaling by routing
Increasing fabric port count and switch count limits meets most customer scaling
requirements. Demand for higher port counts and connectivity between devices in
different fabrics or VSANs requires Fibre Channel routing.
Routing improves scaling by connecting independent fabrics or VSANs, each
potentially at its full capacity. Connectivity between fabrics or VSANs enables you to
share resources, reducing unnecessary redundancy in the routed network.
You can route between fabrics without affecting the total switch and port count limits.
However, the routed network is not the same as a single large fabric or VSAN. Only
selected devices in each fabric, specified by a routing table, can communicate with
devices in other fabrics.
For example, using a router, you can connect three 1,200-port fabrics to construct a
3,600-port Meta SAN.
UC434S F.00
9 -45
Fibre Channel routing implementations
Fibre Channel routing implementations

With Fibre Channel routing, you can create a routed fabric in one of two ways:
Connecting several fabrics using a router
Dividing a single fabric into several smaller VSANs or Virtual Fabrics
There are four Fibre Channel routing techniques:
A B-Series MP Router connects independent fabrics (SAN islands)
A B-Series Virtual Fabric using a virtual backbone switch.
A C-Series switch with IVR connects multiple VSANs
A H-Series switch using a TR_Port
B-Series fabric groups

In B-Series configurations, devices in different fabrics can be grouped together to
form LSANs. In the top graphic the slide shows three fabrics called Fabric 1, Fabric 2,
and Fabric 3, and in each fabric contains one or more switches. Any B-Series switch
can be used in these fabrics. In each fabric, the switches must run the same version
of switch firmware and must have the same variable settings (for example, R_A_TOV
and E_D_TOV). Each fabric has a unique set of fabric services.
Fabrics connected by an MP Router must comply with configuration rules for a routed
fabric. The fabrics can have identical domain names and zoning definitions. The MP
9 -46
UC434S F.00
SAN extension
Router also provides Fibre Channel over IP (FCIP) capabilities, allowing

implementation of Fibre Channel routing and FCIP SAN extension.
Brocade Virtual Fabrics

When you divide a chassis into logical switches, you can designate one of the
switches to be a base switch. A base switch is a special logical switch that is used for
interconnecting the physical chassis. A base switch can be connected to other base
switches through a special ISL, called a shared ISL or extended ISL (XISL). This allows
for other logical switches that share a fabric id to communicate with each other that
maybe in a separate physical switch.
C-Series fabric division

The bottom graphic in the slide shows VSAN 1, VSAN 2, and VSAN 3, each a set of
switch ports on one or more C-Series switches. A VSAN can extend across multiple
switches. Each VSAN has a unique set of fabric services with independent fabric
management. VSANs can share devices by using the license-enabled IVR function.
IVR is distributed across all switches in the SAN, and there is no separate router
hardware. Because the switches are a connected set, they must run the same version
of switch firmware.
UC434S F.00
9 -47
B-Series and C-Series routing differences
B-Series and C-Series routing differences

A Virtual Back bone switch or VSAN Inter-VSAN Routing function can connect
existing fabrics or VSANs. When existing fabrics are connected to a backbone
switch, it creates a Meta SAN. Using C-Series switches, you can physically connect
existing fabrics and configure the routing function in the switches using IVR.
As shown in the graphic on the left In B-series configurations, a switched fabric is
partitioned into several Virtual Fabrics. Each Virtual Fabric has its own set of
resources, such as administrator and users, Name Server, and zoning database.
Devices can be shared across multiple Virtual Fabric administrative domains using
IFR, thus increasing resource sharing without the need for a router.
The graphic on the right shows that Cisco VSANs can include devices that connect to
a single switch or multiple switches in the SAN. Devices in different VSANs can
communicate by using IVR. Multiple switches can be connected in any supported
fabric configuration.
9 -48
UC434S F.00
SAN extension
Basic MP Router configuration
In B-series routing configurations, devices in different fabrics can be grouped to form

LSANs. An LSAN is similar to a Fibre Channel zone, but can extend through a router
to include devices in other fabrics. This configuration, which includes the physical
fabrics (subnetworks), LSANs, and router, is called a Meta SAN. A Meta SAN
consolidates multiple fabrics into a single entity.
UC434S F.00
9 -49
Fibre Channel routing techniques H-Series switch
H-series switch 8 Gb port can be configured as a TR_Port (Transparent Router), which

can be used to connect devices on the H-series switch to devices to a remote fabric.
This is achieved by configuring TR mapping, which establishes a route to connect
one device on the H-series switch to one device on a remote fabric through one
TR_Port. Multiple devices can share TR_Ports, and can be configured for multiple
TR_Ports to the same remote fabric. HP currently supports connection to B-series and
C-series remote fabrics.
9 -50
UC434S F.00
SAN extension
Routing through an IP network

MP Routers connecting fabrics through IP

When connecting fabrics through IP, the MP Router can serve as an FCIP gateway for
Fibre Channel routing. Routers that communicate with the FCIP protocol must be
installed in pairs.
VSANs can be connected through IP using the FCIP functionality of the C-Series Fibre
Channel switches.
UC434S F.00
9 -51
Five-fabric configuration with FC-IP

Five-fabric configuration with FC-IP
Hp have created a specific configuration for remote replication it is called a fivefabric solution, which consists of one fabric dedicated to replication and four fabrics
that are dedicated to I/O between hosts and arrays. The diagram above shows the
configuration using FC-IP for the replication fabric.
In this configuration, the gold and blue fabrics (6 and 7) are dedicated for host I/O.
A separate fabric consisting of the switches (8) and (9) using a single intersite link
transfer all the replication I/O to the remote site. When implementing this solution
using FC-IP or FC-SONET, only one gateway is required at each site.
9 -52
UC434S F.00
SAN extension
SAN island consolidation

SAN island consolidation
MP Router consolidating SAN islands
SAN island consolidation and scaling

The MP Router consolidates SAN islands (multiple independent fabrics) into a Meta
SAN. This modular SAN design offers:
Simplified scalability that allows you to scale a SAN without having to merge
fabrics
Selective sharing of devices in different fabrics so that only devices required for
specific functions are seen across fabrics
Limited sharing or specific times for data migrations and storage consolidation
Ability to access equipment without changing its physical location. Connecting
multiple fabrics to the MP Router enables sharing of devices located anywhere in
the Meta SAN
Ability to connect B-Series fabrics using Secure Fabric OS
Ability to connect B-Series and M-Series fabrics in a Meta SAN
UC434S F.00
9 -53
The MP Router does not merge fabrics, so existing zoning definitions and assigned
domain IDs can be used without modification. Duplicate zoning definitions and
domain
IDs in fabrics are hidden by the MP Router. Fabrics in a Meta SAN can be scaled
without affecting other fabrics. Multiple SANs can be centralized and consolidated
into one Meta SAN, or partitioned into different administrative domains as required.
HP recommends the use of Fabric Manager to simplify management procedures
when implementing an MP Router-based Meta SAN. The slide graphic shows a
typical configuration for SAN island consolidation.
9 -54
UC434S F.00
SAN extension
Integration of Fibre Channel routing and FCIP

Integration of Fibre Channel routing and
FCIP
No single point of failure configuration with

MP Router providing FC routing and FCIP
Integration of Fibre Channel routing and FCIP

You can use the MP Router's integrated FCIP capability to extend disaster-tolerant
applications such as HP Continuous Access for storage arrays.
In typical HP Continuous Access configurations, local and remote fabrics merge
when connected through FCIP. The IP connection is like an ISL in a single fabric. By
using the MP Router Fibre Channel routing feature along with FCIP, the local and
remote fabrics connect without merging. You can create an LSAN that contains local
and remote storage arrays and servers.
The slide graphic shows a typical HP Continuous Access no single point of failure
configuration in which the MP Router provides Fibre Channel routing and FCIP.
UC434S F.00
9 -55
Six-fabric configuration
Six-fabric configuration
The six-fabric configuration consists of two fabrics that are dedicated to replication
and four fabrics that are dedicated to I/O between the hosts and arrays. The
diagram above shows the configuration using FC-IP for the replication fabrics.
As seen previously with the five-fabric configuration there are four local and remote
fabrics two at each site these are represented by (6 and 7) these are dedicated to
host I/O. The way a Six and Five fabric configuration differs is Six-fabric utilizes
separate redundant fabrics which are made up of switches (8 and 10) and two
intersite links (9 and 11). Zoning can be implemented to build the dedicated
replication fabrics out of the local/remote fabrics. In either case, when using physical
or zoned fabrics, a unique gateway is used to connect the local and remote
replication-dedicated fabrics (8 and 10) to the intersite links (9 and 11).
9 -56
UC434S F.00
SAN extension

Tape library

The MP Router enables tape consolidation across multiple fabrics. Increased
consolidation enables tape backup for devices in fabrics without tape libraries. Tape
libraries and backup operations can be centralized and shared across multiple
fabrics in a Meta SAN. There is no need to merge fabrics, thereby reducing
equipment and management costs.
The slide graphic shows a configuration in which an MP Router consolidates tape
backup in a Meta SAN. Independent fabrics connected through an MP Router cannot
have a direct ISL connection. A direct ISL connection between fabrics bypasses the
MP Router, resulting in a full fabric merge.
UC434S F.00
9 -57
Broadcast Zones
Inter-fabric broadcast frames
The FC router can receive and forward broadcast frames between edge fabrics and
between the backbone and edge fabrics. Many target devices and HBAs are unable
to handle these broadcast frames. If devices are connected to B-Series switches which
have a Fabric OS v5.3.0 or later, then a broadcast zone can be set up to control
which devices will receive the broadcast frames. It is to be noted therefore that all
devices that are connected to switches that are running an earlier firmware version
will receive all broadcast frames.
To prevent inter-fabric forwarding of broadcast frames to switches running older
versions of firmware, the fcrBcastConfig command can be used on the FC router. By
default, broadcast frames are forwarded from the FC router to all edge fabrics. The
fcrBcastConfig command can be used to specify which fabrics should not receive the
broadcast frames.
Feature
Verified Limit
Maximum Limit
Device Alias
8000 per fabric
20000 per fabric
Event Traps forward via email
One destination
Up to 10 destinations
VSANs
80 VSANs per physical

fabric
4000 VSANs per physical

fabric
Switches in a single MDS

physical fabric or Vsan
55 switches
239 switches
Switches in a mixed or open

physical fabric or vsan
32 switches
239 switches
Domains per VSAN
40 Domains
239 Domains
Zone members
16 000 zone members per

physical fabric including
VSANs
20 000 zone members per

physical fabric including
VSANs
Zones
8000 zone per physical

fabric including VSANs
8000 zone per physical

fabric including VSANs
Zone Sets
500 zone sets per switch

including VSANs
1000 zone sets per switch

including VSANs
Supported hops
7 hops
12 hops
IVR zone members
4000 IVR members per

physical fabric
20000 IVR zone members per

physical fabric.
IVR zones
1500 IVR zones per

physical fabric
8000 IVR zones per physical

fabric
9 -58
UC434S F.00
SAN extension
IVR zone sets
32 IVR zone sets per

physical fabric
32 IVR zone sets per physical

fabric
IVR Service Groups
16 service groups per

physical fabric
16 service groups per physical

fabric
Certain design considerations must be followed to reach these limits, and should be
validated if being used in a large fabric configuration.
Edge fabric scalability
Domains
Maximum number of front domains per edge fabric
10
Maximum number of EX_Ports per edge fabric
48
Maximum number of translate domains (number of remote fabrics)

per edge fabric
31
Maximum number of real domains (local switches) per edge fabric
26
Maximum number of domains per edge fabric (real domains +

front domains + translate domains)
67
Note: The total number of domains in the first three rows must not
exceed 69.
Device ports
Maximum number of local and remote devices per edge fabric.

(For edge fabrics with more than 600 devices, 4.2.0c or later is
required for all switches in the fabric when using the 400 MP
Router, MP Router Blade, or MP Router.)
1300
Maximum number of imported devices per edge fabric
1000
Meta SAN scalability

Edge Fabrics
Maximum number of edge fabrics connected in a Meta SAN
Meta SAN
Maximum number of total ports per Meta SAN
32
12800
Maximum number of edge fabrics per chassis:

MP Router and MP Router Blade
16
DC Director, 8/80 and 8/40 switches with Integrated Routing
24
Maximum number of EX_Ports per chassis with Integrated Routing

(DC Director, 8/80, 8/40 switches)
Routers
Maximum number of MP Routers per Meta SAN
LSAN
Maximum number of LSAN device entries (proxy devices) per Meta

SAN
Maximum number of LSAN zones per Meta SAN
128/80/40
10
10000
2,500 (5.x)
5,000 (6.x)
UC434S F.00
9 -59
Backbone fabric scalability

Switches
Maximum number of local switches per backbone fabric
32
Domains
Maximum number of translate domains per backbone fabric
42
Maximum number of total domains per backbone fabric

WWNs
Maximum number of local WWNs per backbone fabric
512 (32 edge

fabrics)
1,280 (1 edge
fabric)
Hop Count
9 -60
Maximum number of hops between switches (including routers) in a

Meta SAN
12
UC434S F.00
FCoE/CEE
Module 10
Objectives
UC434S F.00
10 -1
FCoE (Fibre Channel over Ethernet)

CEE (Converged Enhanced Ethernet
FCoE (Fibre Channel over Ethernet)

CEE (Converged Enhanced Ethernet
FCoE overview
FCoE terminology
Layer 2 Ethernet overview
FCoE Initialization Protocol
FCoE queuing
CEE map
DCBX (Data Center Bridging eXchange Protocol)
Fibre Channel over Ethernet (FCoE) is a standard proposed, and developed by the
InterNational Committee for Information Technology Standards (INCITS T11). FCoE
already has been given various names by partners in the industry, such as Cisco,
Brocade and IBM, names that have been given so far are: Data Center Enhanced
Ethernet, Converged Enhanced Ethernet or Converged Enterprise Ethernet.
The concept of the proposal as its name implies was to map Fibre Channel natively
over Ethernet and allowing it to be independent of the Ethernet forwarding schema.
This allows for an evolutionary approach towards I/O consolidation by preserving all
fibre Channel constructs, maintaining the same latency, security and traffic
management attributes of /fibre Channel while still preserving the investments
already made with in the Fibre Channel environment.
The aim of FCoE is to simplify storage environments by using Ethernet, but avoiding
the need to create a separate protocol for I/O consolidation and being able to
leverage the Fibre Channel technology. This is achieved by the Fibre Channel frame
being encapsulated into an Ethernet frame, but it is critical to resolve the acceptance
of the Ethernet of packet loss, to make it become a lossless fabric, and replacing the
Fibre Channel link with a MAC address. In theory FCoE can be broken down into
three components:
Encapsulation of a Native Fibre Channel Frame into an Ethernet frame
The extension of Ethernet to become a lossless fabric
10 -2
UC434S F.00
FCoE
The replacing of a Fibre Channel link with MAC addresses in a lossless Ethernet
fabric
UC434S F.00
10 -3
FCoE/CEE
FCoE/CEE
Eliminate extraneous cabling and hardware
Reduce operational costs and capital
Simplify IT management and reduce

complexity
Reduce the carbon footprint by reducing

power in the data center
evolutionary approach towards I/O consolidation

by preserving all fibre Channel constructs
Fibre Channel over Ethernet (FCoE) enables for the transportation of FC protocols
and frames over Converged Enhanced Ethernet (CEE) networks. CEE is an enhanced
Ethernet that enables the convergence of various applications in data centers (LAN,
SAN, and HPC) onto a single interconnected technology.
FCoE provides a method of encapsulating the Fibre Channel (FC) traffic over a
physical Ethernet link. FCoE frames use a unique EtherType that enables FCoE traffic
and standard Ethernet traffic to be carried on the same link. FC frames are
encapsulated in an Ethernet frame and sent from one FCoE-aware device across an
Ethernet network to a second FCoE-aware device. The FCoE-aware devices may be
FCoE end nodes (ENodes) such as servers, storage arrays, or tape drives on one end
and FCoE Forwarders on the other end. FCoE Forwarders (FCFs) are switches
providing FC fabric services and FCoE-to-FC bridging.
The motivation behind using CEE networks as a transport mechanism for FC arises
from the desire to simplify host protocol stacks and consolidate network interfaces in
data center environments. FC standards allow for building highly reliable, highperformance fabrics for shared storage, and these characteristics are what CEE
brings to data centers. Therefore, it is logical to consider transporting FC protocols
over a reliable CEE network in such a way that it is completely transparent to the
applications. The underlying CEE fabric is highly reliable and high performing, the
same as the FC SAN.
In FCoE, ENodes discover FCFs and initialize the FCoE connection through the FCoE
Initialization Protocol (FIP). The FIP has a separate EtherType from FCoE. The FIP
10 -4
UC434S F.00
FCoE
includes a discovery phase in which ENodes solicit FCFs, and FCFs respond to the
solicitations with advertisements of their own. At this point, the ENodes know enough
about the FCFs to log into them. The fabric login and fabric discovery (FLOGI/FDISC)
for VN-to-VF port connections is also part of the FIP.
UC434S F.00
10 -5
FCoE I/O Consolidation
I/O Consolidation
Although being simple in concept I/O consolidation where Ethernet and Fibre
channel can share the same physical cable and still maintain protocol isolation, and
utilize and configure the same type of hardware for either network is complex. But
benefits from this simple idea are significant. By leveraging I/O consolidation
organizations will free up slots by using a combined Network Interface Card (Nic)
and a Host Bus Adapter (HBA) providing a multi-function network/san. In turn this
will also reduce power consumption, from the reduced number of cards, which in the
case of PCI Express is 25 watts per card, reduced number of switch ports, plus the
reduction in power consumed in the cooling process; which is the primary barrier to
data-center expansion and inefficiency encountered at the present moment in time.
Another advantage of I/O consolidation is that it will give enterprise organizations
the means to simplify their cable management. At the moment 20 Gb of bandwidth
may be provide by two 4Gb FC Connections and twelve 1Gb Ethernet connections.
By combining Fibre Channel and Ethernet this can be achieved by using two 10
Gigabit Ethernet cables still maintaining the bandwidth but reducing the number of
cables being managed by 75%. This also results in fewer points of management
administrators will have to control.
10 -6
UC434S F.00
FCoE
FCoE
Storage Management and Service Delivery

A key concept of the original Fibre Channel aim was reliability, security, availability
and manageability by being able to keep the existing fibre Channel San based
administrative and management procedures in place allowing them to continue to
work in an FCoE network will ensure a smooth transition into an FCoE environment.
By ensuring compatibility with existing Fibre Channel networks will allow usage of
existing FC SAN tools and management constructs.
From a Storage perspective, zoning is a basic provisioning function that is used to
give hosts access to Storage, ensuring security by denying access to unauthorized
devices to the san. FCoE based switches will continue to offer this zoning function
ensuring that storage allocation and security mechanisms are unaffected.
Another integral element in SANs is the Name Server Database, which provides list
of devices connected to the network including information about location and status.
Implementations of FCoE based SANs will continue to offer the Name Server
Database function.
Enterprise organizations also require services such as Disaster Recovery, Backup and
Recovery. Disaster Recovery solutions depending of the distance of the solution may
use FCIP. FCIP remains unchanged within FCoE solution. Data that needs to be sent
to a remote site will hop on to a FCIP link via a gateway and similarly enter an FCoE
network via a FCIP gateway.
UC434S F.00
10 -7
File Backups to Tape Libraries and Recovery from these devices will continue to use
existing backup and recovery tools.
FCoE is designed to maintain the Fibre Channel model, allowing the utilization of the
same management tools, providing the same views of Fibre Channel initiators and
targets seen in a SAN but being used in FCoE environment.
By encapsulating native Fibre Channel in FCoE, the transparent interoperability of
existing SAN networks is possible, because everything above the transport layer
remains intact. This enables existing virtualization applications to continue to work
within FCoE.
10 -8
UC434S F.00
FCoE
FcoE Terminology
FCoE is designed to enable the transport of storage and networking traffic over the
same physical link, as such ports and devices have to be distinguished from other
devices that are using the infrastructure, such a standard network and fibre channel
entities, which are not FCoE.
UC434S F.00
10 -9
FCoE integrated with FC San fabric
The intermediate switching devices in the CEE network do not have to be FCoEaware. They simply route the FCoE traffic to the FCoE device based on the Ethernet
destination address in the FCoE frame.
For supported configurations and limitations check the HP white paper:
HP StorageWorks Fibre Channel Over Ethernet Application Note
10 -10
UC434S F.00
FCoE
OSI, FCoE and FC stacks

FCoE Stack
FC Stack
ULP
Scsi-3
ULP
Scsi-3
Transport
FC-4
FC-4
Network
FC-3
FC-3
Data Link
FC-2
FCoE Mapping
Mac
FC-2
FC-1
Link
Link
FC-0
OSI Stack
Application
Presentation
Session
Encapsulation
By mapping of FC onto Ethernet, the encapsulation of the Fibre Channel can occur.
Both Fibre Channel and networks have stack layers concept, where each layer
represents functionality within the protocol. Ethernet is typically considered a set of
protocols defined by seven-layer OSI stack that define the physical and data link
layers. The Fibre Channel stack consists of five layers, FC-0 through FC-4, which was
created very much aligned to the OSI model from the physical, up to and including
the transport layer.
FCoE mapping allows FC traffic to pass over an Ethernet infrastructure by providing
the capability to carry the FC-2 layer traffic over the Ethernet layer, which allows for
Ethernet to transmit the upper Fibre Channel layers FC-3 and FC-4 over the IEEE
802.3 Ethernet layers.
UC434S F.00
10 -11
FCoE encapsulation
FCoE encapsulation
FCoE
Ether type
16 bit
4 bit
Version
field
SOF
8 bits
Encapsulated FC frame
100 bit reservation field

Destination
Mac Address
48 bits
IEEE
8.02 Q tag
32 bit
Source
Mac address
48 bits
EOF
FCS
Reserved
Frame Format
FCoE encapsulates a Fibre Channel frame within an Ethernet frame.
The first 48-bits in the frame are used to specify the Destination MAC address; the
next 48-bits specify the Source MAC Addresses. The 32-bit IEEE 802.1Q Tag
provides the same function as it does for virtual LANs, which allow for multiple virtual
networks across a single physical infrastructure. FCoE has its own Ethernet type which
is designated by the next 16 bits; this in turn is followed by the 4-bit version field. The
next 100-bits are reserved and are followed by the 8-bit Start of Frame and then the
actual FC frame. The 8-bit End-of Frame delimiter is followed by 24 reserved bits. The
frame ends with the final 32-bits dedicated to the FCS function which provides error
detection for the Ethernet frame.
The encapsulated Fibre Channel frame consists of the original 24 byte FC header
and the data being transported including the Fibre Channel CRC. The FC header is
maintained so that when a traditional FC Storage Area Network is connected to an
FCoE capable switch the frame are de-encapsulated and handed off seamlessly. This
capability enables FCoE to integrate with existing FC SANs without the need of a
gateway.
10 -12
UC434S F.00
FCoE
Frame size is also a factor in FCoE. A classical Ethernet frame is typically 1.5 KB or
less. To maintain good performance, FCoE must utilize jumbo frames or at least the
2.5 KB baby jumbo frame to prevent a Fibre Channel frame from being split into
two Ethernet frames, as a typical Fibre Channel data frame has a 2112 byte payload,
a header and Frame Check Sequence.
UC434S F.00
10 -13
Lossless Ethernet
Lossless Ethernet
Port A
Port B
Fibre Channel Traffic

BB = 1
BB=0
FC uses BB_Credits to guarantee a lossless fabric
Port A
Port B
Ethernet Traffic
Pause
Queue Full
Ethernet uses PAUSE to guarantee a lossless fabric
Lossless Ethernet
Fibre Channel transport frame is a lossless transport format, where congestion has to
be managed to ensure no data packets are dropped. This is achieved in Fibre
Channel by the use of link-level, credit based flow control that guarantees that frames
are not lost in normal conditions. Ethernet on the other hand, when used with TCP/IP
uses a packet drop flow control to handle congestion and is therefore not lossless, so
making it unacceptable for use with storage traffic. In the IEEE 802.3x standard an
optional PAUSE capability is defined which means a busy receive port can send a
control frame to a transmitting port requesting a pause in transmission. By utilizing
this feature Fibre Channel traffic is able to use an Ethernet network in a lossless fabric
format.
A new Ethernet enhancement being developed will allow the PAUSE functionality to
be enabled for each user-priority supported in Ethernet. While PAUSE provides the
basic functionality to make Ethernet lossless, the new proposal for Priority Flow
Control will provide significant benefit for both FC and Ethernet traffic.
10 -14
UC434S F.00
FCoE
HP Converged network switches offerings
HP 2408 FCoE Converged Network Switch

Features
Next-generation Ethernet L2/L3 switch and next-generation FC switch merged
into one product
CEE support
Full industry-standard implementation
Supports FCoE and FCoE Initialization Protocol (FIP)
24 10-GbE and 8 8-Gb/s FC ports
Uses 1 RU of space
Leverages existing Brocade fabric operating system (FOS)
Supports link aggregation (LAG) on DCB ports
Supports Brocade ISL trunking on FC ports Considerations
The port types are fixed. You cannot use 10-GbE ports for FC connections, and
you cannot use FC ports for 10-GbE connections.
L3 routing features are not currently supported.
10-GbE ports support virtual F_Ports only (virtual E_Ports are not supported).
UC434S F.00
10 -15
HP StorageWorks DC SAN Director Switch 10/24 FCoE Blade

Features
24 10-GbE ports
32 8-Gb/s FC ports on the backplane (DC and DC04 SAN Directors only)
Hot pluggable
Blade power and status LEDs
Link status LEDs for each port
FCoE switching
CEE switching
L2 Ethernet protocols STP/MSTP/RSTP, VLAN tagging, and link aggregation
Standard Ethernet encapsulation
Considerations
The DC SAN Director Switch 10/24 FCoE Blade has 24 FCoE ports and can
be installed in either a DC or DC04 SAN director.
The DC SAN Director Switch 10/24 FCoE Blade enables server edge
connectivity by connecting CNAs directly to any of its 24 Ethernet ports.
Storage can be connected to:
Any other Fibre Channel blade in the same director
Any Fibre Channel switch that is in the same fabric as the DC or DC04 SAN
Director that contains the DC SAN Director Switch 10/24 FCoE Blade
The DC SAN Director Switch 10/24 FCoE Blade supports optical cabling and
SFP+ transceivers only.
There are no licensing requirements for this blade.
HP C-series Nexus 5010/5020 Converged Network Switches
Features
L2 access:
IEEE DCB and FCoE support IEEE DCB is a Cisco unified fabric product with
additional proprietary features.
NX-OS with combined features from Cisco IOS and Cisco MDS 9000 SANOS/NX-OS
Cisco 5020 (2 RU):
40 10-GbE ports
10 -16
UC434S F.00
FCoE
Two optional expansion module slots

Up to 52 10-GbE ports, or a combination of 40 10-GbE ports and 16 FC ports
Cisco 5010 (1 RU):
20 10-GbE ports
One optional expansion module slot
Up to 26 10-GbE ports, or a combination of 20 10-GbE ports and 8 FC ports
Optional expansion modules:
6-port 10-GbE expansion module
8-port 4-Gb/s FC expansion module
4-port 4-Gb/s FC and 4-port 10-GbE expansion module
UC434S F.00
10 -17
Converged Network Adapters (CANs)
HP offers the HP CN1000E CNA and also supports the Emulex LightPulse LP21000
family of CNAs and QLogic 8100 Series CNAs in certain ProLiant servers.
Features
The HP CN1000E CNA has the following features:
Ships with half-height and full-height brackets
Dual ports for redundancy
Full 10-Gb/s bandwidth on both ports
Each port can operate as a NIC and/or FCoE port
2 SFP+ connectors
Supports optical or copper cables
Considerations
x8 PCI Express Gen2 card
Requires 14.5 W of power
1 GbE is not supported
The Emulex CNAs have the following features:
Emulex 4-Gb/s FC dual-port controller
Dual-port 10-GbE Intel Opland 82598 NIC
10 -18
UC434S F.00
FCoE
10-GbE and FC multiplexed through a Cisco Menlo ASIC

Considerations
Requires a full-height/full-length x8 PCIe slot
Requires 25 W of power
The QLogic CNAs have the following features:
Full hardware offload for FCoE protocol processing
Full support for TCP/IP and Ethernet performance enhancements, such as
priority-based flow control (802.1Qbb), jumbo frames, checksum offloads, and
segmentation offloads
Considerations
Low profile PCI Express card
Requires 6.9 W of power
UC434S F.00
10 -19
Ethernet Overview
FCoE hardware contains CEE ports that support FCoE forwarding. The CEE ports are
also backwards compatible and support classic Layer 2 Ethernet networks. In Layer 2
Ethernet operation, a host with a Converged Network Adapter (CNA) can be directly
attached to a CEE port on the FCoE hardware. Another host with a classic 10Gigabit Ethernet NIC can be either directly attached to a CEE port, or attached to a
classic Layer 2 Ethernet network which is attached to the FCoE hardware.
Layer 2 Ethernet frames are forwarded on the CEE ports. 802.1Q VLAN support is
used to tag incoming frames to specific VLANs, and 802.3ac VLAN tagging support
is used to accept VLAN tagged frames from external devices. The 802.1D Spanning
Tree Protocol (STP), Rapid Spanning Tree Protocol (RSTP), and Multiple Spanning
Tree Protocol (MSTP) are used as the bridging protocols between Layer 2 switches.
FCoE hardware handles Ethernet frames as follows:
When the destination MAC address is not in the lookup table, the frame is
flooded on all ports except the ingress port.
When the destination MAC address is present in the lookup table, the frame is
switched only to the correct egress port.
When the destination MAC address is present in the lookup table, and the
egress port is the same as the ingress port, the frame is dropped.
10 -20
UC434S F.00
FCoE
If the Ethernet Frame Check Sequence (FCS) is incorrect, because the switch is
in cut-through mode, a correctly formatted Ethernet frame is sent out with an
incorrect FCS.
If the Ethernet frame is too short, the frame is discarded and the error counter
is incremented.
If the Ethernet frame is too long, the frame is discarded and the error counter
is incremented.
Frames sent to a broadcast destination MAC address are flooded on all ports
except the ingress port.
When MAC address entries in the lookup table time out, they are removed. In
this event, frame forwarding changes from unicast to flood.
An existing MAC address entry in the lookup table is discarded when a device
is moved to a new location. When a device is moved, the ingress frame from the
new port causes the old lookup table entry to be discarded and the new entry
inserted into the lookup table. Frame forwarding remains unicast to the new port.
When the lookup table is full, new entries replace the oldest MAC addresses
after the oldest MAC addresses age and time out. MAC addresses that still have
traffic running are not timed out.
UC434S F.00
10 -21
CEE Map
The following information is needed for CEE configuration:

The types of traffic flowing through an interface, FCoE, TCP/IP, and so on.
The minimum bandwidth required for each traffic type.
Which traffic type needs lossless behavior.
The first step is to define the types of traffic carried over the CEE network.
As an example, servers in the diagram above use the CEE network for both FCoE
and IP. The administrator associates FCoE traffic with priorities 2 and 3 and IP
traffic with priorities 0, 1, and 4-7. All the priorities used for IP traffic are grouped
into a single Priority Group ID titled PGID 1, and the priorities used for FCoE are
grouped into PGID 2.
Bandwidth requirements for each PGID are then chosen. The administrator decides
to give IP traffic 60 percent of the schedule and FCoE traffic 40 percent. Finally,
since FCoE traffic requires lossless communication, PFC is also enabled for PGID 1.
10 -22
UC434S F.00
FCoE
DCBX (Data Center Bridging eXchange Protocol)
DCBX (Data Center Bridging eXchange Protocol) runs on CEE links and is an
extension of the Link Layer Discovery Protocol (LLDP). The primary goal of DCBX is to
allow the discovery of CEE-capable hosts and switches and allow CEE-specific
parameterssuch as those for ETS and PFCto be sent before the link is shared.
The IEEE 802.1AB Link Layer Discovery Protocol (LLDP) enhances the ability of
network management tools to discover and maintain accurate network topologies
and simplify LAN troubleshooting in multi-vendor environments. To efficiently and
effectively operate the various devices in a LAN you must ensure the correct and
valid configuration of the protocols and applications that are enabled on these
devices. With Layer 2 networks expanding dramatically, it is difficult for a network
administrator to statically monitor and configure each device in the network.
Using LLDP, network devices such as routers and switches advertise information about
themselves to other network devices and store the information they discover. Details
such as device configuration, device capabilities, and device identification are
advertised. LLDP defines the following:
A common set of advertisement messages.
A protocol for transmitting the advertisements.
A method for storing the information contained in received advertisements.
UC434S F.00
10 -23
NOTE
LLDP runs over the data-link layer which allows two devices running different network
layer protocols to learn about each other.
LLDP information is transmitted periodically and stored for a finite period. Every time
a device receives an LLDP advertisement frame, it stores the information and
initializes a timer. If the timer reaches the time to live (TTL) value, the LLDP device
deletes the stored information ensuring that only valid and current LLDP information is
stored in network devices and is available to network management systems.
10 -24
UC434S F.00
FCoE
VLAN Membership
IEEE 802.1q Virtual LANs (VLANs) provide the capability to overlay the physical
network with multiple virtual networks. VLANs allow network traffic isolation into
separate virtual networks reducing the size of administrative and broadcast domains.
A VLAN contains end stations that have a common set of requirements which can be
in independent physical locations. You can group end stations in a VLAN even if
they are not physically located in the same LAN segment. VLANs are typically
associated with IP subnets and all the end stations in a particular IP subnet belong to
the same VLAN.
In addition to creating a special VLAN for FCoE traffic, VLAN classifiers are applied
to incoming EtherTypes for FCoE Initiation Protocol (FIP) and FCoE. VLAN classifiers
are rules used to dynamically classify Ethernet frames on an untagged interface to
VLANs.
Traffic from downstream CEE interfaces can be assigned to a VLAN using several
methods:
The VLAN tag contained in the incoming frame
The VLAN classifiers
The Port-VLAN ID (PVID)
Because the Ethernet uplink ports from the Brocade FCoE hardware to the distribution
layer switches will carry traffic for multiple VLANs, they are configured as 802.1q
trunk ports.
UC434S F.00
10 -25
The downstream CEE ports connected to the server CNAs are configured as access
ports with a PVID of either 10 or 20. The VLAN classifier group created for the FIP
and FCoE EtherTypes must be applied to the interfaces in order to place FCoE traffic
on the correct VLAN. The CEE map is also applied to the interface.
Note: Up to 4,000 VLANs, but only one FCoE VLAN is currently supported
10 -26
UC434S F.00
FCoE
Minimum CEE configuration to allow FCoE traffic

flow
FCoE CN switches have dual capabilities in that they serve as both an Ethernet
switch and an FC switch. You must perform a setup procedure to achieve the desired
function.
A 2408 FCoE Converged Network Switch or DC SAN Director Switch 10/24 Blade
can be a standalone switch or an edge switch in a Fibre Channel fabric. To attach
the switch to an existing Fibre Channel fabric as an edge switch, at least one Fibre
Channel port on the FCoE
CN switch must be connected to a Fibre Channel switch in the fabric (E_Port). There
cannot be any other FCoE or 10-GbE CEE switches in the path to the Fibre Channel
switch.
Similarly, a DC or DC04 SAN Director with a 10/24 FCoE Blade installed can be a
standalone switch, an edge switch, or a core switch in a Fibre Channel fabric. If it is
an edge switch or core switch, other switches in the fabric can be attached to any
available FC port (E_Port) on other FC blades in the director.
For FCoE E_Port connectivity, see the appropriate Fabric OS release notes for the
minimum and recommended Fibre Channel switch firmware versions and the
supported Fibre Channel switch models. HP recommends Brocade FOS 6.3.0b or
later.
UC434S F.00
10 -27
A C-series FCoE CN switch can be a standalone switch or an edge switch in a Fibre

Channel fabric. To attach the switch to an existing Fibre Channel fabric as an edge
switch, at least one Fibre Channel port on the FCoE CN switch must be connected to
a Fibre Channel switch in the fabric (E_Port). There cannot be any other FCoE or 10GbE IEEE DCB switches in thepath to the Fibre Channel switch.
For FCoE E_Port connectivity, the Fibre Channel switch minimum firmware version is
NX-OS 4.1(3)N1(1). All C-series 4 Gb/s or 8 Gb/s switch models are supported
when using the minimum firmware version. MDS switches running SAN-OS 3.x can
be in the Fibre Channel SAN but cannot be connected directly to an FCoE switch.
10 -28
UC434S F.00
FCoE
FCIP, ISCSI & FCoE

FCIP, iSCSI & FCoE
FC over IP (FCIP),
iSCSI and FCoE
are all storage
protocols capable
of transporting
block storage data
over Ethernet
FCIP, in contrast, was

designed as a simple
tunnelling protocol to
link Fibre Channel
SANs over distance on
IP networks. Primarily
used for remote storage
access and disaster
recovery, FCIP provides
a SAN-to-SAN
connectivity over IP, it is
important to remember
that each end point is a
FC device
File System
SCSI
FC 4
iSCSI / iSER
TCP/IP
FCoE
FC
Ethernet
iSCSI was designed

to reliably transport
block storage data
over any IP
infrastructure. iSCSI
relies on the entire
TCP/IP protocol
stack at Layers 3
and above to
support routing and
packet recovery, has
the benefit that it can
be used in networks
that potentially could
be described as
lossy.
FCoE is being
developed as a
streamlined data center
storage protocol,
intending to take
advantage of the Layer
2 protocol efficiency of
Fibre Channel and the
data center Ethernet
FCIP, iSCSI & FCoE

FC over IP (FCIP), iSCSI and FCoE are all storage protocols capable of transporting
block storage data over Ethernet. Each has been designed and developed with
different design criteria and goals to solve particular issues with the storage media
environment. FCoE is being developed as a streamlined data center storage
protocol, intending to take advantage of the Layer 2 protocol efficiency of Fibre
Channel and the data center Ethernet. iSCSI was designed to reliably transport block
storage data over any IP infrastructure, including LANs and WANs. iSCSI relies on
the entire TCP/IP protocol stack at Layers 3 and above to support routing and packet
recovery, and so therefore has the benefit that it can be used in networks that
potentially could be described as lossy. FCIP, in contrast, was designed as a simple
tunnelling protocol to link Fibre Channel SANs over distance on IP networks.
Primarily used for remote storage access and disaster recovery, FCIP provides a
SAN-to-SAN connectivity over IP, but it is important to remember that each end point
is a FC device. FCIP like iSCSI carries the overhead of TCP/IP processing, which is
essential component for maintaining data integrity over long-distances. FCIP can be
used to create disaster recovery solutions that can link FC Sans that can be up to
distances of 20 000 KM (12 500 miles) apart, this depends on the latency of the
link.
UC434S F.00
10 -29
A key benefit of iSCSI is its ability to integrate with an existing network environment,
utilizing Network interface cards, switches and routers to transport SCSI storage
block data between servers, desktops and even laptops to storage media. Although
touted as inexpensive solution, iSCSI storage targets costs vary depending on the
type of disks that are used is it hardware based or software based implementation.
As there are no iSCSI disk drives, ISCSI is reliant upon some form of bridging
protocol iSCSI to SATA, SAS, or Fibre Channel controllers to retrieve and store
block data. It has to be noted that iSCSI is an affordable means to integrate a lower
performing storage in to a 1Gbit/sec Ethernet providing shared storage for
departmental use. At 10 Gbit/sec, iSCSI loses its much of its publicized cost
advantage. By using a 10Gbit/ sec Ethernet implies that the applications being
hosted require high reliability and performance. At 1Gbit/sec standard NICs can
be used, however when implementing on 10Gbit/sec network server performance is
enhanced by the use of iSCSI cards which utilize auxiliary components like TOE (TCP
off-load Engine) or iSER (iSCSI Extensions for RDMA), which helps to avoid multiple
memory copies of SCSI data between the interface and application memory. These
types of cards (TOE and/or iSER) can add significant cost per attached server
compared to an 8Gbit/sec FC HBA, and could undermine the value proposition of
iSCSI at 1 Gbit/sec.
By the use of iSCSI-to-FC gateways would enable iSCSI initiators to access FC
storage targets, the required protocol conversion is more complex than FCoE frame
mapping to Fibre Channel. When using an iSCSI gateway, a complete address
translation is required between the iSCSI and FC address. In addition, the gateway
must act as proxy virtual FC initiators and also a virtual iSCSI targets plus it must also
terminate sessions within the gateway between the iSCSI and fibre Channel
protocols. If the ultimate objective is to have Ethernet-attached servers accessing FC
SAN targets, then FCoE will require less protocol overhead and processing latency to
span between Ethernet and Fibre Channel transports.
FCoE is a component technology that enables highly efficient movement of block
storage over Ethernet for consolidating server network connectivity. By enabling
organizations the opportunities to deploy a single server interface for fibre channel
and Ethernet traffic. By simplifying the management and deployment infrastructure of
server network connectivity, and still maintaining the high available and reliable
standards required for storage data transactions. FCoE is to be viewed not as a
replacement for, but an extension of, Fibre Channel and is intended to coexist with
existing FC SANs.
10 -30
UC434S F.00
FCoE
Storage Support
HP storage system support for access from CNA-based servers. These storage systems
can be attached to Fibre Channel switches in the fabric or connected to the Fibre
Channel ports of an FCoE CN switch.
SAN boot is not supported when storage systems are connected to the Fibre Channel
ports of an FCoE CN switch. SAN boot is supported for storage attached to the Fibre
Channel ports in a SAN Director that contains a 10/24 FCoE blade and based on
the current support listed for Fibre Channel switches.
UC434S F.00
10 -31
Operating System Support
FCoE is supported on operating systems above:

Boot from SAN support
BFS is not supported for FC arrays attached to 2408 FCoE Converged Network
Switches or C-series Nexus 5000 Converged Network Switches at this time.
BFS is supported for FC arrays attached to standard FC switches in a fabric that
includes CN switches.
DC SAN Director Switch 10/24 FCoE Blade supports BFS from the storage attached
to any of the other FC blades in the same director or to any Fibre Channel switch
that is part of the same fabric as the DC SAN Director that contains the 10/24 FCoE
Blade.
10 -32
UC434S F.00
SAN Management
Module 11
Objectives
Objectives
Discuss the need for SAN management
Technologies driving SAN management
HP SAN management today
HP Storage Essentials
UC434S F.00
11 -1
Storage management tasks

Storage management tasks
according to Gartner
Security
Storage Resource
Allocation and Usage
Storage Area Discovery

Asset Management
Device Insertion, Removal
Perfomance Planning
Capacity Planning
SAN Design
Device Management
Manage Other Tools
Escalation Management
Diagnosticvs and Probes
SAN
SAN
Storage Capacity
QoS Planning
Intuitive Console
Backup Planning and Management
Policy Based Administration
Reporting and Billing
Security Administration
Task Automation
Provisioning
SAN
NAS Management
SAN
SAN
SAN
System Availability
Bandwidth
Management
Bulk Data Delivery

Source: Gartner, Automating SAM:
A Manifestation, Nick Allen
According to a Gartner Study, there are a lot of dedicated Storage Management

tasks, which of course may have different priorities depending on the customer:
11 -2
Storage Area Discovery
Asset Management
Device Insertion, Removal
Performance Planning
Capacity Planning
SAN Design
Device Management
Manage Other Tools
Escalation Management
Diagnostics and Probes
QoS Planning
Intuitive Console
Backup Planning and Management
Policy Based Administration
Reporting and Billing

UC434S F.00
SAN Management
Security Administration
Task Automation
Provisioning
NAS Management
UC434S F.00
11 -3
11 -4
UC434S F.00
SAN Management
Storage Resource Management

Storage Resource Management
Cost recovery
Volume
management
Disaster
management
Provisioning
Access and Security

management
Application and patch

management
Resource
management
Fault
management
Performance
management
Device
management
Storage Resource Management gains vital importance by providing the information

and tools that are necessary to manage, maintain and grow the heart of the IT
infrastructure.
Shifting perspectives on enterprise storage
As storage challenges continue to grow, perspectives on enterprise storage
management are beginning to shift. In the past, storage was viewed from a point
perspective, typically as an add-on direct attached storage (DAS) device for
individual servers. But because the demand for storage is exploding, a
comprehensive integrated storage strategy is necessary to efficiently and costeffectively meet growing enterprise needsa strategy that maximizes the use of
existing resources and reduces the administrative burden so that more storage can be
managed by the same number of administrators.
Beyond efficiency gains, the integrated storage service management strategy allows
customers to manage, monitor, and view storage resources and services as an
integral component of the overall management solution. An important characteristic
of this strategy is an understanding of the impact of storage on other management
component and services, as well as the overall business objective.
UC434S F.00
11 -5
SAN management categories

SAN management categories
SAN fabric management

Control of network traffic
Device communication
Interconnect components
SAN storage management

Control of the specific storage system configuration
SAN data management

Applications that ensure data is available and accessible
SAN usage and monitoring

SAN event notification and failure information
Report and billing information
SAN management is wide ranging, covering many aspects of the day-to-day

activities used for monitoring, managing, and simplifying the complexity of the
storage network.
SAN management can be classified into four major categories:
Fabric management
Storage management
Data management
SAN usage and monitoring
Fabric management
SAN fabric management can be thought of as the control of the SAN infrastructure
or traffic flow within the SAN. This concept pertains to control and management of
device communication or access within the SAN, such as switch zoning, or LUN
Masking. This concept also includes managing SAN interconnect components,
individually and collectively, throughout the fabric.
11 -6
UC434S F.00
SAN Management
Storage management
Storage management allows control of the specific storage system configuration such
as redundant paths, creation and management of storagesets (LUNs), the setting of
RAID levels, and the setting of platform-specific SAN interface characteristics and
parameters.
Data management
SAN data management applications help ensure that data is available and
accessible. The data being stored on the SAN is part of a company's assets. It is
imperative to keep this data available to system applications with minimal, if any,
downtime. Techniques such as cloning, snapshots, data replication, and backups
protect the data from disasters.
Usage and monitoring

SAN and storage usage and monitoring applications are necessary to provide SAN
event notification and fault and failure information for service before SAN anomalies
can adversely impact the enterprise. They can also provide reporting and billing
information for determining the amount of storage and quality of service delivered.
UC434S F.00
11 -7
SAN management
SAN management
The HP SAN management strategy
Simplify storage management using standardized web-based
GUIs
Centralize the management of multi vendor heterogeneous SANs
Automate policy-based management
Optimize functionality by exploiting all currently available
management levels
HP is rapidly transitioning from the traditional server, storage, and component levelbased management to SAN-level application architecture and implementation using
the HP recommended Microsoft Windows server running SAN and Storage
management software.
Just as important as the quality and feature set of the SANs hardware is the
effectiveness of the SAN management applications in tying these devices together
and simplifying the complexity of the storage network. Whether using an HP
standard topology or a custom design using the HP StorageWorks SAN Design
Guide rules, IT managers must configure, monitor, and maintain the SAN, as well as
plan for and accommodate growth.
The HP Open SAN Management strategy is to:
11 -8
Simplify storage management using standardized web-based graphical user

interfaces (GUIs) residing on easy-to-use, easy-to-implement storage
management appliances
Centralize the management of multivendor heterogeneous open SANs in
distributed and consolidated environments
Automate policy-based management
Optimize functionality by exploiting all available management levels such as
appliances, SAN fabrics, and servers and storage
UC434S F.00
SAN Management
SAN performance management

SAN performance management
It is important to understand:
Managing performance to meet service levels
SAN fabric load monitoring and leveling
The behavior of applications
Optimizing the network
Performance data collection
I/O and disk performance management
Storage performance data collection
I/O performance analysis
Monitoring and analyzing SAN fabric traffic patterns
64
Today, larger SANs surface within IT infrastructures as businesses progress to grow

their SANs. In these cases that congestion can become real problem. Congestion
results in serious problems, such as system reboot, lost data and slow response times,
which in the end is inability to fulfill storage service level agreement (SLA).
Fire fighting approaches to reducing congestion are expensive and can result in
making the situation worse. Unlike SAN device management, which is individual,
SAN performance management requires an integrated view of the whole network.
Traffic flows across the entire SAN and involves all SAN devices. Understanding the
traffic patterns on the network, through the collection of traffic matrices, is the
essential first step in a performance management strategy.
A SAN architect must understand:
Managing performance to meet service levels
SAN fabric load monitoring and leveling
The behavior of applications
Optimizing the network
Performance data collection
I/O and disk performance management
Storage performance data collection
UC434S F.00
11 -9
I/O performance analysis
Monitoring and analyzing SAN fabric traffic patterns
If users experience performance problems in the system, you can use the
performance collection methods to find out where the problem existed or to eliminate
the problem areas. The problem might seem related to distance, however, the
customer might think the issues is the system and the problem might have nothing to
do with the system, which had 80% availability the whole time.
Performance data is used to support these kinds of incidents and problems. From a
storage perspective, it is important to register, monitor, and collect the performance
of the logical volumes that are offered to the systems.
11 -10
UC434S F.00
SAN Management
Storage capacity management

Storage capacity management
The planning, acquisition, and optimal usage of SAN
resources
Understanding that SAN capacity planning is a primary
SAN management function
SAN trend analysis has become an important part of
storage capacity management
The following components impact analysis
Availability
Service level
Hardware and software components
Topology configuration
Workload and application traffic
65
To ensure a well designed SAN, the SAN architect must research capacity planning
for future requirements. The SA is also responsible for contingency planning, storage
capacity analysis, and regression forecasting.
The following areas comprise storage capacity management in corporate
management environments:
The planning, acquisition, and optimal usage of SAN resources that are driven
by agreed service levels at lowest possible cost.
Understanding that SAN capacity planning is a primary SAN management
function and one of the main goals is long-term planning of approximately one
to three years in the future. It is also important to consider short-term planning
goals of approximately one to five months in the future.
SAN trend analysis, which has become an important part of storage capacity
management with regression reporting providing necessary information that can
then be used for the SAN performance management process. Trend analysis can
also be utilized for future SAN usage prediction and prediction of SAN service
levels.
The impact analysis of the following components during SAN topology changes.
These factors must be included in capacity planning and management:
Availability
Service level
UC434S F.00
11 -11
Hardware and software components
Topology configuration
Workload and application traffic
11 -12
SAs use of the capacity planning what if model to simulate configuration

changes based on real data, application traffic workload changes, and
saturation analysis and utilize extended SAN modeling.
UC434S F.00
SAN Management
SMI-S
SMI-S
Replaces multiple disparate managed object models,
protocols, and transports with a single object-oriented
model for each type of component in a storage network
Created by the SNIA
Enables management application developers to support
devices from multiple vendors
SMI-S components:
Common Information Model (CIM) (object model)
WBEM
Service Location Protocol (SLP) (discovery protocol)
Organized around profiles that describe objects relevant

for a class of storage subsystem (arrays, HBAs, and
SAN devices)
53
UC434S F.00
Replaces multiple disparate managed object models, protocols, and transports

with a single object-oriented model for each type of component in a storage
network
Created by the SNIA
Enables management application developers to support devices from multiple
vendors
SMI-S components:
Common Information Model (CIM) (object model)
WBEM
Service Location Protocol (SLP) (discovery protocol)
Organized around profiles that describe objects relevant for a class of storage
subsystem (arrays, HBAs, and SAN devices)
11 -13
Implementing SMI-S
Implementing SMI-S
SMI-S is implemented with:

CIM server (called a CIM Object
Manager or CIMOM)
CIM provider
Components provided by
vendors as:
Embedded agent
SMI-S solution
Managing Storage with SMI-S

The movement to standardize storage networking management is being directed by
the Storage Networking Industry Association (SNIA) and its Storage Management
Initiative (SMI). The SNIAs mission is to advance the adoption of storage networks
as complete and trusted solutions. The SNIA is a vendor neutral trade organization
that works with its end user, vendor and channel members to make storage
networking technologies understandable, simpler to implement, easier to manage
and recognized as valued assets to the business process.
The idea for the SMI-S was born in 2000 to help end-users alleviate the pain points
associated with implementing and managing multi-vendor storage infrastructures.
SMI-S v1.0.2, the current version of the SMI-S, is the first step towards making multivendor storage networks simpler to implement and easier to manage. The
specification delivers a reliable interface that allows storage management systems to
identify, classify, monitor and control physical and logical storage resources.
Additionally, the SMI-S program has reached important milestones in its
unprecedented rise to prominence:
11 -14
More than 100 storage products have emerged into the market place that
conforms to the SMI-S v1.0.2.
UC434S F.00
SAN Management
Vendor products have passed the SNIA Conformance Testing Program (SNIACTP), which means they are conformant to the SMI-S interface. For the first time
in the history of the storage industry, end-users and integrators can ask for
vendor products that conform to a functionally rich, open, secure and extensible,
storage management interface standard.
SMI-S enabled products deliver a value chain that includes simpler configuration
and set-up procedures, standardized controls for complex operations and
automated provisioning.
Presently, the SMI-S is the only standard that addresses the end user's need for
reducing the costs associated with multi-vendor storage management. Today, each
device in the SAN has its own disparate management interface, which is a
nightmare for administrators and systems integrators. As the first standard to
addresses consolidating the management interfaces, it aims to reverse the trend of
vendors developing their own proprietary management approaches and contributing
to the storage management nightmare.
At a high-level, SMI-S is a standard focused on management interoperability between
storage hardware providers and software management application clients.
Essentially, SMI-S provides a standardized management interface to enable
"management interoperability" for storage hardware and software.
Reducing the Mistakes of Day-to-Day operations

The scope of SMI-S includes virtually the entire set of devices that connect to the
SAN. In its initial form the specification covers fundamental operations of the
communication between management console client and the devices including autodiscovery, access, security, the ability to provision volumes and disk resources, LUN
mapping and masking and other active management operations. Version 1.0.2 of
the specification covers the day-to-day activities of the storage administrator.
The SMI-Lab, located at the SNIA Technology Center in Colorado Springs, is the
center where vendors implementing SMI-S collaborate to improve the quality of the
standard and test management interoperability.
SMI-S is designed to be extendable over time, and is presently focused on managing
storage devices, arrays, tape libraries, switches and HBAs in the storage network.
SMI-S v1.0.2 currently addresses fiber channel devices and their management clients.
Today, future versions under development in the SMI-Lab include IP based storage
networks such as NAS and iSCSI.
SMI-S is being implemented for individual devices as an SMI-S server, which is a
mechanism for representing hardware devices to management applications. There
are several implementation models for providing this service. The most often used
model is to create a proxy, an implementation that provides one or more SMI-S
profile for use by a management application and all devices of a certain type. With
this interface, a single management application can manage all devices of the same
type, regardless of the manufacturer and without using proprietary APIs.
UC434S F.00
11 -15
The SNIA-Conformance Test Program (SNIACTP) is the testing process to validate the
implementation of the SMI specification and assure that it conforms to the SNIA
standard in this case SMI-S v1.0.2. The testing process is a critical building block in
the effort to make multi-vendor storage operate in a predictable manner for end
users. Products that pass the SNIA-CTP offer IT Administrators more confidence in the
technology they are purchasing reducing the risk when deploying complex
networked storage solutions.
Storage vendors that successfully complete the SNIA-CTP master test suites receive a
formal confirmation and are allowed to use the SNIACTP mark. This mark indicates
that the storage vendor has completed conformance testing and can be placed on
product packaging and in marketing materials.
End-users looking to ensure that a vendors SMI-S implementation conforms to SNIA
standards should look for officially marked and tested products and they can check
the SNIA-CTP site: (http://www.snia.org/tech_activities/sniactp/certified) for
specific details.
On the horizon are standardized storage management products that have
implemented features to the specification standard. These products will offer
increased trust and flexibility in choosing solutions that reduce the overhead and
complexity of managing storage.
11 -16
UC434S F.00
SAN Management
Storage Essentials
Storage Essentials
Base components
Enterprise Edition
Discovery
Event management
Discovery
Event Management Capacity
Capacity
Role-based
security
Role
-Based Security
Topology
Performance
Path
management
Historical
&
future
trends
Topology
Path Management
Performance
Historical & Future Trends
Reports
Tool kit
forKit
custom
scripts
Reports CLI /APIs
CLI / APIs
Tool
for Custom
ScriptsPolicy
Policy Manager
Manager
Base functionality provided by

System Manager
Capacity Manager
Performance Manager
Application Viewer
Policy Manager
Event Manager
Reporter
Business Tools (Enterprise Edition only)
The Storage Essentials base components listed above provide essential functionality
and require purchase of the Storage Essentials base product.
An Oracle database 10g Release 2 (10.2.0.3) database is bundled with the base
Storage Essentials product (both editions). This database is controlled by Storage
Essentials and cannot be modified for other use, nor can it be remotely deployed by
another server.
In addition to the basic components, plug-in Storage Essentials modules are available
with the product or included for a fee. Storage Essentials software is required to
support any of the other modules.
UC434S F.00
11 -17
Feature
Application
storage
management
Configuration
management
Reporting
11 -18
Component
Description
Database Viewer
Database (Oracle, SQL, Sybase or InterSytems Cache)

availability and performance views
Exchange Viewer
Microsoft Exchange database availability and

performance views
File System Viewer
Performs a recursive lookup on the file system and stores

the information in an embedded database
Provisioning
Manager
Heterogeneous host-to-array path provisioning wizard
Chargeback
Manager
Assign tiers and create asset-based chargeback

management
NAS Manager
Report (disc, topology, capacity, performance, etc.) and

event management of NAS
Backup Manager
Backup configuration, management, and reporting
Global Reporter
View rollup reporting of multiple Storage Essentials

instances
Report Designer
Develop customer reports for your storage infrastructure
UC434S F.00
SAN Management
Storage Essentials Enterprise Edition plug-ins
All of the above are plug-ins and are available within Storage Essentials Enterprise
Edition. Most are licensed feature this can be based on ports, application, or
storage capacity.
UC434S F.00
11 -19
Storage Essentials Enterprise Edition home page
The Enterprise Edition supports several additional plug-ins (modules) not available
with the Standard Edition. Licenses are needed for the added plug-ins. At the time of
this writing, upgrade from Standard Edition to Enterprise Edition is not supported.
11 -20
UC434S F.00
SAN Management
Description of base components

Description of base components
These base components are common to both editions of
Storage Essentials
System Manager
Capacity Manager
Performance Manager
Application Viewer
Policy Manager
Event Manager
Reporter
Each is described in the following slides
The following pages will introduce the base components of HP Storage Essentials.
UC434S F.00
11 -21
System Manager
The System Manager has two panesthe tree pane and connections pane.
The System Manager is often the first tool used to access the managed infrastructure
to investigate element details collected and stored within the SE database.
The connections pane contains a graphical representation of the SAN topology
including discovered applications, hosts (with CIM extension agents), storage
switches, storage arrays and tape libraries (based upon supported products).
The tree pane, on the left side, contains the following three tabs
11 -22
List Elements by fabric and alphabetical name
Access Elements by zone entry, host binding, and LUN access
Path Elements by server
UC434S F.00
SAN Management
Capacity Manager
The SE Capacity Manager provides a graphical representation of element capacity

and utilization. Topology maps appear similar to System Manager, but do not
provide the same level of object browsing as the System Manager. The navigation
tabs for the Capacity Manager are:
List Elements by fabric
Path Elements by host
Utilization Host, switch, subsystem utilization summary reports
Capacity and utilization reports show changes over time for hosts, switches,
applications, and storage.
UC434S F.00
Host utilization Mounted and un-mounted disks and percentage of used

mounted disks
Switch capacity Ratio of used-to-unused switch ports
11 -23
Performance Manager
The SE Performance Manager provides a graphical representation of the

performance history for a managed element. Charts in the pane on the right side
can be manipulated to show a different reporting period and frequency
Performance Manager provides:
Detailed performance metrics for managed hosts, switches, subsystems, and

applications
Flexible interface to monitor historical and real-time performance data
Metrics that change based on selected elements
Multiple windows for viewing different performance metrics and different time
periods
Similar to the System and Capacity Managers, the Performance manger provides
topological view of the connected infrastructure for ease of management.
Performance enabled objects are represented by additional symbols on the map.
11 -24
UC434S F.00
SAN Management
Application Viewer
The SE Application Viewer helps you navigate the following types of applications:
Exchange applications
Oracle applications
SQL applications
Sybase applications
InterSystems Cache
DB2
Informix
Application instances with File System Viewer enabled
Virtual applications
Storage Essentials application discovery features allow a DBA, for example, to

understand how database tables are affected by the storage infrastructure. Storage
administrators will be able to monitor the use of disk capacity associated with the
managed applications. Tablespace to LUN topology mapping allows for an end-toend view of the application.
UC434S F.00
11 -25
Policy Manager
The SE Policy Manager helps you efficiently manage both utilization and
infrastructure policies. Navigation is in the left pane and display is in the right pane.
Policy Manager enables you to:
11 -26
Set utilization watermark on elements
Root node policies apply to all elements under that node (element type)
Host policies apply to all mount points on that host
Individual policies for mount points (if necessary)
Establish system policies
New element discovery
Provisioning action (Enterprise Edition)
Events
UC434S F.00
SAN Management
Event Manager
Event Manager lets you view, clear, sort and filter events from managed elements. An
event can be anything that occurs on the element, such as a device connected to a
Brocade switch has gone off-line. It provides the following information about the
events:
ID Identification number assigned to the event

Element Source of the event. An element can be a switch, host, application,
fabric or anything else on the network.
Severity Severity level
Time Time the event was recorded
Summary Text Brief explanation of the event. When you click the summary
text, the details of the event are displayed.
Element Type - Specifies whether the source of this event is an application, a
host, and so on
Rank By estimated cost implication. This column is hidden until you enable it.
Storage Essentials device events are also visible within the HP SIM event
management structure.
UC434S F.00
11 -27
Report Optimizer
The SE Report Optimizer provides a variety of detailed reports, such as dependency,

event, and utilization reports, for discovered elements. To view a report, click a report
name in the tree in Reporting. The report appears in the right pane.
The Standard Edition software provides reports for the following:
11 -28
System These reports are enterprise wide and they collect information about
the following:
Application Data about applications the management server monitors, such
as reports on application utilization and dependencies.
Events Data about events occurring on the elements the management server
monitors, such as summary reports on events.
Fabric Data about fabrics, such as SAN components not zoned and world
wide names that appear in zones but not in SANs.
File Server Data about the file servers the management server monitors, such
as reports on groups and users by server. This information is provided only if you
have purchased the license for File System Viewer.
HBA A summary report on the host bust adapters (HBAs) the management
server detects.
Host Data about the hosts in the management server monitors, such as
reports on host storage allocation and total host utilization.
UC434S F.00
SAN Management
UC434S F.00
NAS Data about NAS storage devices, such as reports on volume and
aggregate usage.
Performance Historical performance data for devices, such as reports on I/O
performance.
Storage System Data about storage systems the management server monitors,
such as reports on storage system capacity and storage system utilization.
Switch Data about switches the management server monitors, such as reports
on switch port traffic and port utilization by connection type.
Backup Manager Data about backups, such as reports about the status of the
daily backup, backup volume and media availability.
Applications These reports provide information about an application, such as
Oracle, SQL, Sybase, or Microsoft Exchange.
Tape Libraries These reports provide information about a tape library.
Recent Lists the last 10 reports viewed. This option is not displayed when you
first access Reporting.
Only Enterprise Edition has support for the following reports:
Global These global-wide reports provide data gathered from multiple
management servers.
Asset Management These reports provide information based on assets and
ownership.
Chargeback Manager These reports provide cost information about the
management and storage usage of an element.
11 -29
Description of plug-ins for both editions

Description of plug-ins for both editions
These plug-in modules are common to both editions of
Storage Essentials
Database Viewer
Exchange Viewer
File System Viewer
NAS Manager
Backup Manager
Each is described in the following slides
11 -30
UC434S F.00
SAN Management
Database Viewer
Within the SE Application Viewer, the Database Viewer offers application-to-spindle

management of an Oracle, SQL, Sybase database, or InterSystems Cache
databases. With its powerful capacity, performance, and dependency management
capabilities, Database Viewer helps ensure the availability and performance of
Oracle, SQL, or Sybase databases and applications in SAN environments.
For all databases, Database Viewer:
UC434S F.00
Automatically discovers the database, tablespaces, devices, and files and then
graphically depicts their dependencies on the SAN.
Contains a single, integrated view depicts the path that each database element
takes through mount points, host servers, volume management software, host bus
adapters (HBAs), fabric switches, and storage systems. These path management
capabilities, combined with real-time performance monitoring features, enable
you to predict the impact of planned and unplanned SAN downtime on
database applications and determine where in the SAN stack performance is
being impacted.
Extends the full range of capacity management, role-based security, event
management, reporting, and policy-based automation capabilities of Storage
Essentials to database environments.
11 -31
Exchange Viewer
The Exchange Viewer automatically discovers Exchange Server instances, storage

groups and message stores, and graphically depicts their dependencies on the SAN.
The path that each Exchange resource takes through mount points, Windows hosts,
volume management software, HBAs, fabric switches, and storage systems is
represented in a single, integrated view. These path management capabilities,
combined with real-time performance monitoring features, enable you to predict the
impact of planned and unplanned SAN downtime on your messaging environment,
and determine where in the SAN stack performance is being impacted.
The Exchange Viewer also extends the full range of capacity management, rolebased security, event management, reporting, and policy-based automation
capabilities of HP Storage Essentials Suite to Exchange Server environments.
The HP Storage Essentials Exchange Viewer monitors the performance from message
stores through the SAN to storage system disks. The Exchange Viewer discovers and
highlights messaging dependencies on Windows hosts, HBAs, fabric switches, and
storage systems.
11 -32
UC434S F.00
SAN Management
File System Viewer
File System Viewer extends the Storage Essentials suite with scalable file-level storage
resource management (SRM) capabilities. Whether the file servers are hosted on
direct attached storage (Enterprise Edition), NAS, or SANs, File System Viewer
provides the file system scanning, analysis, and reporting features you need to
reclaim wasted disk space, ensure file server availability, monitor user consumption,
and classify unstructured data for ILM initiatives.
File System Viewer discovers file systems, logical volumes, and user shares, and
conducts high performance scans to collect age, size, and type statistics on every
file. Detailed reports categorize files according to file extensions (for example, mp3,
log, tmp, pst), size, and dates created, last accessed, and last modified so you can
quickly identify disk space that can be recycled and critical files that should be
replicated. Because the module is tightly integrated with the SAN management
capabilities of Storage Essentials, it also depicts file server dependencies on HBAs,
fabric switches, and storage systems, and extends Storage Essentials full range of
role-based security, event management, reporting, and policy-based automation
capabilities to file servers.
File System Viewer reports on logical volumes, files, user shares, and user and group
disk space consumption File type analysis quickly identifies inappropriate or
unnecessary files that can be deleted or archived to reclaim disk space.
UC434S F.00
11 -33
Backup Manager
Backup Manager
Supports
HP Data Protector
Veritas NetBackup
Monitors the overall
status of the backup

process
Visualizes the backup
configuration (topology)
Views the status of the
physical infrastructure
supporting the backup
process, backup
application, backup
server, network, tape
library, and media
Backup Manager monitors your backup applications running on discovered hosts.

Backup Manager enables you to do the following for HP Data Protector and Veritas
NetBackup:
11 -34
Monitor the overall status of the backup process; shows health and performance
of the data protection infrastructure. Identifies unprotected applications, servers,
files. Increases utilization of backup resources.
Visualize the backup configuration and recoverability of a file, directory, volume
or a server.
View the status of the physical infrastructure supporting the backup process,
backup application, backup server, network, tape library and media.
Provide information on reasons for backup failures and advisory information for
configuring new backup schedules.
UC434S F.00
SAN Management
HP StorageWorks Fabric Manager

HP StorageWorks Fabric Manager
Client-server management
application for:
Levels
Medium to large B-Series fabrics

Multiple B-Series fabrics
All supported HP B-Series switch
types
SAN
Management tool
Storage
Essentials
Java-based
Monitor and configure multiple
switches and perform SAN-level

maintenance from one location in
real-time
Integrated with Advanced Web
Tools for individual switch

management
SAN/Fabric
B-Series
Element
Fabric
Manager
Web Tools
What is StorageWorks Fabric Manager?

Fabric Manager is a highly scalable client-server management application that
enables you to manage one or more B-Series fabrics simultaneously from one
location in real-time. Through its single-point SAN management platform, Fabric
Manager enables the global integration and execution of processes across multiple
fabrics.
The software is a Java-based host application (not switch based). It leverages many
B-Series Fabric OS services such as web tools and zoning. Fabric Manager is not
tied to a specific B-Series Fabric OS firmware or switch hardware release.
The current edition includes the following enhancements:
Improved user interface
Firmware management
Tool integration
Device diagnostics wizard
Improved reporting
Centralized RADIUS management
UC434S F.00
11 -35
For a list of the latest supported B-Series switches and firmware versions, refer to the
HP StorageWorks SAN Design Reference Guide available from:
http://h18000.www1.hp.com/products/storageworks/san/documentation.html
Fabric Manager is tightly integrated with the entire family and can extend those
products' capabilities (such as Web Tools and Fabric Watch). This unique ability to
work tightly with all B-Series management tools effectively reduces the time and costs
of managing B-Series fabrics. From Fabric Manager you can launch the Advanced
Web Tools application for a specific switch to perform element (switch) management.
This action is transparent to the user. Fabric Manager provides more efficient fabric
management than Advanced Web Tools alone.
To download a copy of Fabric Manager, go to:
http://h18006.www1.hp.com/storage/networking/b_switches/index.html
11 -36
UC434S F.00
SAN Management
Key features and functions (1 of 2)

Discovery
Topology visualization
ISL monitoring
Fabric merge checking
Multiple device login
Firmware and configuration download
Events and status
Fabric, switch, and port administration
MP Router and EX_port management

Note: Some Fabric Manager features run only on a particular firmware version. Refer
to the HP StorageWorks Fabric Manager User Guide available from
http://docs.hp.com for details.
UC434S F.00
Discovery Discovers details about devices logged in to the fabric.

Topology visualization View the SAN layout through a topology map that
specifies inter-switch link (ISL), switch, and device details.
ISL monitoring Retrieve and save ISL information on the fabric, including
trunking information, and monitor ISL performance.
Fabric merge checking Looks for merge compatibility across zoning, security,
and other potentially conflicting areas.
Multiple device login Simultaneously log in to multiple devices (switches and
MP Router).
Firmware and configuration download Download across fabrics to all B-Series
switch types and firmware versions.
Events and status Monitor, filter, and display events and provide reasons for
various statuses.
11 -37
11 -38
Fabric, switch, and port administration
Launch Advanced Web Tools to perform element management
Fabrics, switches, and ports can be named
Switches and ports can be enabled and disabled
Switch login credentials are saved for a session, so that users only need to
authenticate themselves for a switch or multiple switches once
MP Router and EX_port management
Share devices between two or more fabrics
Configure EX_Ports on one or more MP Routers
Display logical SANs and information
UC434S F.00
SAN Management

SAN element grouping
Sequenced reboot
Fabric-wide supportShow
Change management
Performance monitoring
Secure fabric management
License management and e-licensing
Persistence
Call home
UC434S F.00
SAN element grouping Group multiple HP switches or ports in the same

fabric to perform tasks on the group simultaneously. For example, you can:
Enable or disable all switches in the group.
Download firmware to all switches in a group with the same model number.
Log in to a group of switches at one time.
Activate license keys on a group of switches.
Sequenced reboot Create and save a sequence of rebooting groups of

switches in a predetermined order. The reboot group must contain one or more
switches from the same fabric. A switch can belong to one reboot group only.
This is useful to:
Simultaneously reboot switches that run the same firmware.
Simultaneously reboot switches of the same model.
Reboot the core switches of a fabric and then the edge switches.
Reboot distant physical locations sequentially.
Fabric-wide supportShow Save supportShow command output for one or

more switches in a fabric as text files. These can be used for troubleshooting.
11 -39
11 -40
Change management Monitor specific changes in the fabric by taking

snapshots at scheduled intervals and comparing the snapshots to a baseline
snapshot. You can export XML versions of the change reports.
Performance monitoring Capture port traffic and store it in the Fabric
Manager database, then generate reports and graphs.
License management and e-Licensing After you make a request to the HP
license key generation website, license keys are distributed automatically to
multiple switches in the SAN.
Persistence Some application-specific data is persisted across sessions, such
as fabric, switch, port and group names, fabric and group memberships, reboot
sequences, and existing license keys.
Call Home Using the client-side GUI you can configure conditions that trigger
a call-home action. The server monitors changes and events. Requests for actions
are sent based on configured parameters.
UC434S F.00
SAN Management
Graphical user interface


The point-and-click GUI provides an easy-to-use management tool for linking SAN
objects across fabrics using the B-Series Multi-protocol Router. Use the ID menu to
customize how you want to view the switches and fabrics. You can view by:
Element name
IP address
Domain ID
WWN
Use the address field to enter the switch IP address to discover a switch. All
discovered elements display in the SAN Elements navigation tree. These are
organized by fabrics, SwitchGroups, and switch PortGroups. Select an element to
show element details on the view screens. Right-click the element to launch actions.
The background color of each element indicates the physical status of that element:
Blue indicates the element is unknown.
White indicates the element is healthy.
Yellow indicates the element is marginal.
Red indicates the element is critical (down).
Light gray indicates the element is unmonitored.
UC434S F.00
11 -41
Orange indicates the element is missing.
Dark gray indicates the element is user disabled.
If you notice a color change, expand the navigation tree to determine the source.
Note: Port status does not affect switch icon background color.
11 -42
UC434S F.00
SAN Management
Firmware upgrade
Firmware upgrade
Upgrade multiple switches simultaneously across fabrics
using firmware download function

Not supported for MP Router
Requirements:
All switches can run firmware version to be downloaded
TCP/UDP Ports 20 and 21 are available between server and
each switch
To simultaneously reboot switches after download they must
reside on same fabric
Could experience switch connection interruption during
download resulting in time out with no error
Firmware upgrade
You can upgrade switch firmware on multiple switches simultaneously, and across
fabrics, using the Fabric Manager firmware download function. There a certain
requirements that must be met:
All switches must be able to run firmware version to be downloaded.
TCP/UDP Ports 20 and 21 must be available between server and each switch.
To simultaneously reboot switches after download, they must reside on same

fabric.
If you are upgrading firmware from Fabric OS v3.0.0 to v3.1.0, or from Fabric OS
v4.0.0 to v4.1.0, any port name changes that you have made in Fabric Manager are
lost; this ensures that multiple Fabric Manager clients that are simultaneously active
during the firmware upgrade do not overwrite each others port names.
The Fabric Manager firmware download to multiple switches feature is not supported
for switches running XPath OS. If you attempt to download firmware to one of these
switches using Fabric Manager, the Web Tools-AP Edition is launched. See the Web
ToolsAP Edition Administrators Guide for more information.
UC434S F.00
11 -43
Note: If the switch loses network connectivity during the firmware download from
Fabric Manager, the firmware download action times out after approximately 25
minutes for switches running Fabric OS v2.x or v3.x, and after approximately 80
minutes for switches running Fabric OS v4.x or v5.x. An error message is not
returned when the firmware download is interrupted.
11 -44
UC434S F.00
SAN Management
Configuration upload and download

Save switch configuration
Download configuration from baseline file or another switch
across fabrics
Useful to:
Propagate configuration to a new switch before adding to
fabric
Propagate configuration to another fabric before merging two
fabrics
Compare configurations for troubleshooting segmented fabrics

You can save a switch configuration to Fabric Manager as a baseline, and use this
baseline file to compare other switch configurations, or download to another switch.
When you compare the configuration of a switch to a baseline, Fabric Manager
identifies and lists all parameters that do not match.
This capability is useful when you need to:
Propagate a configuration to a new switch before adding it to the fabric
Propagate a configuration to another fabric before merging two fabrics
Compare configurations for troubleshooting segmented fabrics
UC434S F.00
11 -45
HP Data Center Fabric Manager (DCFM)

DCFM provides a topology-centric view of entire data center

fabrics, from servers to storage.
HP Data Center Fabric Manager Enterprise is the latest management solution

from Brocade. A network management solution that enables end to end
management of a storage data center. From storage ports all the way to the
Host Bus Adapters (HBAs) attached to physical or virtualized servers.
HP StorageWorks Data Center Fabric Manager Enterprise Software builds upon
the recognized capabilities of the previous EFCM (McData) and the Brocade
Fabric Manager products and extends it from roots in SAN management to the
data center fabric supporting enhanced scalability, multi-protocol fabrics (Fibre
Channel, iSCSI, FCIP) and policy-based service management including adaptive
networking with Quality of Service features
Data Center Fabric Manager (DCFM) is a Brocade product, it is Licensed by HP
and is available via Brocade or HP on a free 75 day trail basis.
11 -46
UC434S F.00
SAN Management
Utilize Administration Time

Utilize Administration Time
configures the QoS rules for applications to ensure that

critical applications receive priority in the event of
network latency or errors.
monitors user-defi ned events and takes proactive
actionsfrom alerts to correctionsto maintain network
availability and health
provides detailed monitoring of port and link utilization
against predefined thresholds to optimize network
performance and maximize application availability.
UC434S F.00
Data Center Fabric Manager allows for the management and the securing of
data flow across B-Series switches FOS 5.0v or higher, also supporting fabrics in
the interop mode with M-EOS based fabrics, however Fabrics composed of both
FOS and M-EOS require B-Series switches to be running FOS version 6.0 or
higher and M-EOS version 9.6 or higher.
HP DCFM allows for the measurement and the displaying of real-time and
historical network performance, this highly-scalable application Scalability which
can support up to 24 SANs, 9,000 switch ports, and 20,000 end devices
provides the essential functions for efficiently configuring, monitoring, and
dynamic provisioning SAN fabrics through a Enhanced GUI with wizard driven
operations for automating tasks. It also features easy-to-use admin tools that
streamline or automate repetitive tasks so organizations can achieve high levels
of productivity, by configuring Quality of Service (QoS) priorities for improved
utilization of virtual machines, DCFM also supports Brocade encryption
capabilities for data-at-rest and HBA products. This product is available on 75
Day Free Trail download
11 -47
SUPPORTED CLIENT/SERVER OPERATING SYSTEMS

Microsoft Windows 2003 Server, XP, Vista, and 2008 Server
Red Hat Linux AS 4.0, Red Hat Enterprise Linux 5 Advanced, and
SUSE Linux Enterprise Server 10 SP1
Sun Solaris 10
VMware ESX Server 3.5 (Guest OS supported: Windows 2003 Server, Linux Red
Hat AS5, and SUSE Linux ES10)
HARD DRIVE STORAGE

1 GB free disk space is needed for installation. After installation, 5 GB free disk
space for small and medium-sized SANs, and 10 GB for large SANs is needed.*
* To allow for a larger amount of history data, 20 GB (small and medium-sized
SANs) and 40 GB (large SANs) of free disk space is recommended.
11 -48
UC434S F.00
SAN Management
Cisco Fabric Manager overview

Cisco Fabric Manager overview
Two network management tools that supports SNMPv3 and
legacy versions.
GUI
Displays real-time views of your network fabric

Lets you manage the configuration of Cisco MDS 9000 Family
devices and third-party switches
Cisco Fabric Manager tools are:
Fabric Manager
Device Manager
Cisco Fabric Manager

The Cisco Fabric Manager:
Is a set of two network management tools that supports Secure Simple Network
Management Protocol version 3 (SNMPv3) and legacy versions.
Provides a graphical user interface (GUI) that displays real-time views of your
network fabric, and lets you manage the configuration of Cisco MDS 9000
Family devices and third-party switches.
The Cisco Fabric Manager tools are:
Fabric Manager
Device Manager
UC434S F.00
11 -49
Fabric Manager
Fabric Manager
The Fabric Manager displays a view of your network fabric, including Cisco 9000
family or third-party switches and devices. To launch the Fabric Manager from your
desktop, double-click the Fabric Manager icon.
Changes made using Fabric Manager are applied to the running configuration of the
switches you are managing and the changes may not be saved when the switch
restarts. After you make a change to the configuration or perform an operation (such
as activating zones), the system prompts you to save your changes before you exit.
The Cisco Fabric Manager is an alternative to the command-line interface (CLI) for
most switch configuration commands.
For information on using the CLI to configure a Cisco MDS 9000 Family switch, refer
to the Cisco 9000 Family Configuration Guide or the Cisco 9000 Family Command
Reference.
11 -50
UC434S F.00
SAN Management
Device Manager
Device Manager
Tabs
Menu bar
Toolbar
Switching
or services
modules
Supervisor
modules
Physical representation of switch
chassis
Launch by:
Double-click Device Manager

icon
Double-click a switch in the
Fabric Manager topology view
Device Manager
Device Manager provides a physical representation of your switch chassis, with the
modules, ports, power supplies, and fan assemblies. The menu bar at the top of the
Device Manager window provides access to options, organized into menus that
correspond to the menu tree in Fabric Manager.
The legend at the bottom right of the Device Manager indicates port status, as
follows:
Green The port is up.
Brown The port is administratively down.
Red The port is down or has failed.
Gray The port is unreachable.
To launch the Device Manager:
UC434S F.00
From your desktop, double-click the Device Manager icon.

From Fabric Manager, right-click the switch you want to manage on the Fabric
Manager map and click Device Manager from the pop-up menu that appears.
Double-clicking on a switch in the Fabric Manager topology view.
11 -51
11 -52
UC434S F.00
SAN Management
UC434S F.00
11 -53
Lab activity
11 -54
UC434S F.00
SAN Management
Lab
activity
Module 11, Lab 1 - Data
Center Fabric Manager
DCFM
45
UC434S F.00
uc434s c.01
uc434sDevelopment
c.01 2009Company,
Hewlett-Packard
L.P.
Development Company, L.P.
11 -55
11 -56
UC434S F.00
SAN Security
Module 12
Objectives
Objectives
Discuss the basic storage security model and access
points
Describe approaches to planning security in a SAN
Outline the core components for securing SAN Data and
SAN Management
Security in practice
Authentication
Encryption
UC434S F.00
12 -1
Security in a SAN
Security in a SAN
A fundamental requirement for enterprise SANs
Multi-Customer environments have new security
requirements
Security enables sharing of SAN resources among
multiple customers securely
Reduces xSP (multi-customer) infrastructure costs and
enables economies of scale
12 -2
UC434S F.00
SAN Security
Basic security model

Basic security model
Three types of attacks corresponding to the three aspects
of information security
Data can be made unavailable for access
Data can be deleted or modified without permission
Data can be examined without permission
Security can be implemented at three levels in the SAN

Storage array level
Fabric level
Host level
Information security is a fundamental issue that must be dealt with while managing
any data center. HP understands the importance and complexity of establishing and
maintaining a secure information storage environment. HP storage products are
designed to make it easy to protect the availability, integrity, and confidentiality of
the customer data that they hold.
HP is working with other storage vendors in the Storage Networking Industry
Association to develop enhanced SAN security technology.
HP is also working with the Fibre Channel standards community to develop storage
network security protocols.
The ideal mass storage system provides fast storage and retrieval of information for a
number of servers.
This one-line summary leaves unspoken many of additional expectations:
UC434S F.00
It is expected that data written to the storage system today will be available
tomorrow.
It is expected that the data will be the same when it's read as it was when it was
written.
It's expected that the data is not available to any server or any person not
specifically authorized to have access.
12 -3
These three possibilities are covered under the general headings of availability,
integrity, and confidentiality. These additional expectations form the basis for
defining the availability and security of the data in the mass storage system. For
example, the data should be available even if a hardware or software component in
the storage system fails; RAID and remote mirroring technology are methods used to
maximize data availability.
Security is a fundamental requirement for enterprise SANs. Multi-Customer
environments have new security requirements. Security enables sharing of SAN
resources among multiple customers securely and reduces xSP (multi-customer)
infrastructure costs and enables economies of scale.
Security can be implemented at three levels in the SAN:
Storage array level
Fabric level
Host level
Three types of attacks, corresponding to the three aspects of information security, can
be made on a computer system:
Data can be made unavailable for access.
Data can be deleted or modified without permission.
Data can be examined without permission.
Any computer security system must deal with these types of attacks.
The security of a computer system is the responsibility of a security manager. This
person defines the operational rules and procedures that are required to maintain the
desired security level. To achieve the desired security level in an HP SAN system, the
operational rules and procedures should incorporate the guidelines discussed in this
module.
12 -4
UC434S F.00
SAN Security
Security domains
Security domains
Define one or more security domains to make a storage
infrastructure secure
A logical grouping of related components
Include a set of rules that specify the amount of
communication that is allowed
Types
Host-to-switch domain
Administrator-to-security management domain
Security management-to-fabric domain
Switch-to-switch domain
The basic approach to making a system secure is to define one or more security
domains. A security domain is a logical grouping of related components in the
storage system, along with a set of rules that specify the amount of communication
that is allowed between the components. Devices, such as servers and storage
systems that are within a given security domain, are allowed to communicate with
each other. The security manager defines the communicationif anythat is
allowed between domains. The security system works by controlling every possible
communication path between the security domains, so that data cannot be moved
between domains without authorization.
The boundaries of the security domains are barriers that control access to the
components. The boundaries also control communication between domains through
the network or storage bus connections. Any potential path between security
domains must be reviewed to make sure that only approved access is permitted. This
can be an extremely complex undertaking.
Domain types
UC434S F.00
Host-to-switch domain between host servers and their host bus adapters (HBAs),
and the connected switches
Administrator-to-security management domain between administrators and their
management applications
12 -5
12 -6
Security management-to-fabric domain between management applications and

the switch fabric
Switch-to-switch domain between interconnected switches
UC434S F.00
SAN Security
Attacks and exposures

Attacks and exposures
The goal of storage security is protecting:
Confidentiality of data (reads, other than by the
application or user who owns it)
Integrity of data (modifications, other than by the
application that owns it)
Destruction of or loss of access to data, without

authorization
The consequences of these are clear. Sensitive business or customer data can be
exposed, and business records can be altered or destroyed. One can easily imagine
a worst-case scenario for ones own organizationbut also a more typical case,
such as a minor administrative error on one system destroying data belonging to
another.
The goal of storage security is protecting:
Confidentiality of data (reads, other than by the application or user who owns it)
Integrity of data (modifications, other than by the application that owns it)
Destruction of or loss of access to data, without authorization
UC434S F.00
12 -7
Mitigation of risk
Identification (authentication)
An administrator must log on (give his or her own user id, then prove he or she is
that user by knowing a password or using some more sophisticated mechanism)
before administrative actions are permitted.
Emerging technology: a device must not only be on the list of devices permitted in the
storage network, but must also proving that it is in fact who it says it is rather than an
impostor. This prevents a rogue system from, for example, pretending to be a switch
and issuing unauthorized I/Os with forged WWNs to bypass LUN-level security.
Fibre Channels FC-SP protocol works this way; iSCSI accomplishes the same end but
in a slightly different way.
Authorization
12 -8
Storage devices must verify that the specific administrator who issued a
command is authorized to do so, before performing the requested action.
Disk arrays must verify that the specific system that issued a read or write
command has permission to do so for that LUN, before performing the I/O.
Emerging technology: a tape library controller can similarly verify permissions
on I/Os to a tape library.
UC434S F.00
SAN Security
Audit
The storage subsystem as a whole must log all administrative actions (changes)
and any events of significance. This is typically done individually in devices, but
software to present a single view (and allow queries) is preferred.
Encryption (not yet in widespread use)
Protects both confidentiality (no one saw it) and integrity (no one changed it) of
data.
Data on disk can be encrypted.
Data on tape and other removable media can be encrypted.
UC434S F.00
Data in flight between data centers, typically HP StorageWorks Continuous

Access data passing over a WAN connection, can be encrypted to protect
against wiretapping. In the future, data in flight within a data center can also
be encrypted.
12 -9
SAN security access points

SAN security access points
Authentication methods for

SAN access points
User
Management
Server
Switch
To prevent unauthorized
configuration require
multilevel passwords
Extensive use of ACLs
Centralization of fabric
configuration changes on
trusted switches
Authentication methods for SAN access points include:
User access to the management interface
Management console access to the fabric
Server access to the fabric
Switch access to an existing fabric
To prevent unauthorized configuration or management changes, a secure fabric

operating system (OS) employs a policy that uses:
12 -10
Multilevel passwords
Extensive use of Access Control Lists (ACLs)
Centralization of fabric configuration changes on trusted switches
UC434S F.00
SAN Security
Storage security model

Storage security model
The diagram above places major parts of these mitigations into the categories of
data security and management security, then further divides those categories. Some
items in these categories are in routine use today, while others represent the leading
edge of what can be done (or could be done in the next few years). For example,
selectively showing each system only the devices and LUNs it is allowed to access is
a feature in widespread use in SAN installations today, while storage encryption is a
leading-edge technology. Simplified language has been used: Authentication of
users encompasses not just Single Sign On but also more traditional approaches.
UC434S F.00
12 -11
Data security
Data security
No authentication
Fibre Channel WWN
FC-SP
iSCSI
No authorization
LUN Masking
iSCSI
NAS
ILM
Data in transit
between data centres
Data in transit within
the data centre
Data on disk and tape
Authentication
Authentication of devices is an effort expended by a device to ensure the identity of
another device with which it is communicating.
There are three levels of authentication relevant to storage: none, trusting the devices
address, and challenging the device to prove its identity. Historically no
authentication at all was done. More recently the Fibre Channel WWN has been
trusted as a devices identity. Looking to the future, both in Fibre Channel and in
iSCSI, state-of-the-art challenge/response protocols will be used to confirm a devices
identity.
No authentication
Devices on a SCSI cable are presumed to belong there; there is no concept of
identity.
Early Fibre Channel installations split a SAN into zones. A system connected to a
zone was presumed to belong there, much as a system on a SCSI cable. Zoning
remains important for isolating traffic for interoperability or fault isolation reasons.
Fibre Channel WWN

Current Fibre Channel best practice is that online storage identifies the unique WWN
of the system making a request, using that WWN as its identity. For security against
simple administrative errors and against casual attacks this is sufficient. It is
12 -12
UC434S F.00
SAN Security
theoretically possible for a determined and knowledgeable attacker to forge

(spoof) a WWN belonging to another system. Leading-edge mitigation of this
theoretical attack today involves (1) ensuring the SAN contains no unauthorized
rogue switches, (2) disabling all unused ports on the fabric, and (3) restricting each
SAN port to allow only traffic from the WWN (device) that is supposed to be
connected, using an advanced switch feature.
Fibre Channel Security Protocol (FC-SP) (Future)

Current work in the Fibre Channel industry standards body includes developing the
FC-SP security standard, which is hoped to complete in 2004, with product to follow.
Under FC-SP, Fibre Channel devices will mutually authenticate using state-of-the-art
challenge/response protocols. Several years will pass before FC-SPenabled devices
will be pervasive enough to make it practical to lock non-FC-SP devices out of a preexisting SAN.
iSCSI
iSCSI provides for the use of the Challenge Handshake Authentication Protocol
(CHAP, a state-of-the-art challenge/response protocol) for a storage client to
authenticate itself to the storage server, during login time, much as FC-SP does. It is
thus not possible for a storage client to masquerade as a valid user of anothers
iSCSI port ID. However, if the iSCSI traffic is not encrypted, a sophisticated attack
could theoretically take over an established connection. Such encryption is
accomplished using Internet Protocol Security (IPsec), a set of protocols that allows
encryption of data over an IP network like a LAN or even the Internet. IPsec prevents
this attack because the attacker cannot know the correct data encryption keys. IPsec
depends on the customers security infrastructure, specifically on CHAP (or possibly
SRP) for authentication, on IPsec policies, and on an appropriate mechanism for
exchanging keys.
When implementing IP-based storage, whether iSCSI or NAS, it is important to keep
in mind just how broadly a network is connected. While the greatest risk may be
from a disgruntled employee or simple error, an external attack would find a Fibre
Channel SAN contained entirely within a locked data center a much more difficult
target than an open network reaching every desk in a company, which is in turn
harder to reach than the Internet.
Central directories and challenge/response protocols background

While details vary and can be quite complex, state-of-the-art authentication is simple
in concept. A data center (preferably the whole organization) has a single server
containing a list of all the authorized entities (people, applications) in the
organization, and what their roles are (really what they are allowed to access or do).
Assume device A asks device B to perform an action. Device B challenges device A
by giving it a random number to combine with device As key (password) in a
special way. Device B then sends both the random number and device As response
to the authentication server, which responds yes device A used its key, its who it
says it is or no it didnt. Most importantly, this process centralizes authentication
UC434S F.00
12 -13
both without keys ever appearing on the network in the clear and without device B
seeing device As key.
A variant of this approach uses Public Key Encryption, in particular Certificates. For
Public Key Encryption, a particular user (or company or device) is given a pair of
keysa public key and a private key. Two basic operations can be performed:
encrypt data with your private key, and anyone can decrypt it with your public key;
anyone can encrypt a message with your public key, knowing that only you have the
private key to decrypt it. Roughly speaking, for trusted communication between two
users (devices), the message is encrypted using the recipients public key, and then a
checksum of the message is encrypted using the senders private key. A Certificate
is a users name, public key, an expiration date, and the assertion by a certificateissuing authority that it was really the user and not someone else the certificate was
issued to. Care must be taken in which certificate-issuing authorities to trust and in
the actual issuing of certificates: unlike a central directory, which can instantly revoke
an identity, a certificate has a life of typically a year. The whole topic of Public Key
Infrastructurecertificate-issuing authorities, secure distribution of private keys to
their owners, means of finding someones public keyrequires both expertise and
effort to establish and operate.
Authorization
Authorization has evolved from DAS model of if you can see it, you own it to more
sophisticated mechanisms that enable pooling of resources on the SAN.
No authorization needed
As mentioned before, SCSI does not have an authorization mechanism: any system
can read and write any device connected to the same cable.
Early Fibre Channel SANs offered a variant of SCSI by dividing the SAN into
segments, called zones. Each zone behaved like a SCSI cableany system in the
zone could read and write any device in the zone. Later versions allowed
overlapping zones. Today, of course, zoning remains important, primarily used to
isolate traffic for interoperability or fault isolation reasons.
LUN masking/selective LUN presentation based on WWN

Current state-of-the-art authorization for Fibre Channel SANs is that each storage
device maintains, for each LUN it presents to the SAN, a list of which systems (which
WWNs) are allowed to access that LUN. When a system asks the storage device
which LUNs it offers, the storage device responds naming only the LUNs that system
is allowed to access. Likewise when systems sends a read or write I/O to storage, the
storage device checks to see if that system is authorized to perform that read or write.
(Note: this explanation is simplified and several variants of this basic approach are
used by different products in the industry). The LUN authorization lists are typically
created by system administrators who access the arrays configuration utility by way
of a password-protected interface.
12 -14
UC434S F.00
SAN Security
iSCSI
iSCSI devices offer both device level and per-LUN Access Control Lists (ACLs). PerLUN ACLs are similar to Fibre Channel LUN masking. VLANs on the network are
analogous to Fibre Channel zones. It is up to administrators to verify that a particular
array supports the features they plan to use.
Network Attached Storage (NAS)

NAS is delivered by a server connected to the network, behind which either DAS
storage or SAN-connected storage is used. Data accesses are typically by one of two
protocols, either NFS (UNIX heritage) or CIFS (Microsoft Windows
heritage). Both protocols treat permissions for file access in much the same way as
file access on a local system works: a file system has an owner, which sets
permissions (read, write, and so on) based on the users identity and groups to which
the user (identity) belongs.
Mainstream NFS protocols are in widespread use and quite effective. However, they
are subject to spoofing attacks, where an unauthorized entity impersonates an
authorized one, and should always be isolated from malicious users (and from the
Internet) by an appropriately configured firewall.
Devices that combine NAS and iSCSI are starting to become available. While
sharing a common network connection (and hence being behind a common
firewall), the two protocols have very different access rights mechanisms that are
administered separately.
Reference stores and ILM

An emerging class of storage addresses information that is no longer changing but
the organization to store cost effectively for long-term read access and records that
must be stored in compliance with laws or regulations such as those prohibiting
tampering or deletion. Like NAS, this storage can be modeled as a server in front of
DAS storage; such storage is usually instantiated as a cluster of many storage units,
which are accessed by way of emerging protocols over Ethernet LAN. Interestingly,
such stores can include a selection of different types of storage systems that
contribute different performance, availability, and other capabilities to the store.
It is clear that permissions to access archives of corporate trade secrets must be more
restrictive than permissions to access historic press releases. For example, attorneys
doing discovery in e-mail archives need access which for privacy reasons would not
be granted to everyone.
Reference store security standards have not yet emerged, but it is clear that reference
stores (which themselves might contain more than one storage subsystem) will need
security analogous to conventional storage arrays, including both authentication and
authorization. Also, audit trails must be maintained within reference stores. At the
least, these must be able to satisfy regulatory compliance requirements.
UC434S F.00
12 -15
Encryption
Encryption has drawn a lot of attention. Taken purely as a technology, that exotic
branch of mathematics, which for centuries was out of reach of all but military and
espionage, has suddenly found a mass market. The Internet has made it necessary to
secure transactions across an un-trusted connection between parties who trust each
other, and VLSI technology has made such security affordable. On the other hand,
exponential growth in computer power has made it possible for experts to try every
possible key and decode messages, which just a few years ago were thought
unbreakable.
While encrypting (or decrypting) data at hundreds of megabytes per second
storage system speedsis considerably more difficult than encrypting a few
thousand bytes using software on your PC, such speeds are attainable using
commercially available technology today. Many people across the storage industry
have thought about how this technology could be applied, resulting in a number of
products from a variety of companies.
Rather than seeking applications for one of these products, consider systematically
the customer needs that encryption might address. In general, data can be encrypted
either in flight (crossing a Fibre Channel, Ethernet, or WAN network) or at rest (on a
disk or tape).
Data in transit between data centers

When data is copied between data centers, usually as part of a disaster recovery
plan, it is no longer protected from wiretapping by the physical security of those data
centers. There are degrees of risk to this: passing through a few kilometers of optical
fiber in cable channels owned entirely by the customer is a far lower risk than
passing over a leased line, which is in turn a far lower risk than passing the data
over the Internet backbone.
In any of these situations, the lack of physical security on cables outside the data
center can be mitigated by passing the traffic through an encryption box before it
leaves the sending data center, and of course through a corresponding decryption
box after entering the receiving data center. Such boxes are available today for both
Fibre Channel and IP networking, with the latter called IPsec gateways. Such
installations are uncommon today because cost and complexity of such measures are
greater than the perceived risk.
In the particular case of iSCSI, HP anticipates IPsec will be built into future interfaces,
making encryption more affordable and more ubiquitous than is possible with IPsec
gateways, presuming that IPsec policy and key distribution infrastructure are
available. IPsec software layered above standard VLSI is available today; HP
anticipates the industry will have premium-priced interface VLSI with IPsec within a
few years, but that mainstream-priced interfaces will not include IPsec hardware for
at least five years.
12 -16
UC434S F.00
SAN Security
Data in transit within a data center

While the security plans of most data centers establish a secure perimeter and
assume that the risk of wiretapping within the data center is low, situations do exist
where encrypting even traffic within the data center is necessary. In these situations
today, the only available technology is encryption boxes for Fibre Channel (akin to
IPsec gateway boxes). There are two future technologies of interest:
Fibre Channel Security Protocols (FC-SP future)
The Fibre Channel Security Protocols (FC-SP) standard includes not just authentication
as previously mentioned, but also Encapsulating Security Payload (ESP) encryption,
which provides a way for Fibre Channel devices to exchange keys and then encrypt
all data flowing between them. Because all elements of the SAN must have not just
FC-SP but also the encryption feature before there is a real benefit to its use, HP
anticipates it will be several years before ESP is used and many years before it is
pervasive.
iSCSI (IPsec future)
IPsec is in common use today, although rarely at the speeds needed for storage. As
Ethernet interfaces with built-in line speed encryption become more common (and
less expensive), it will become practical to encrypt storage traffic in the data center.
This is at least several years in the future.
Data on disk (online storage)

Incidents in which media containing valuable data, or customer data which must
remain private, are stolen from a data center are reported regularly in the media.
Privacy legislation such as California SB1386 makes these incidents more visible,
except when the data is encrypted.
Data is encrypted by transforming it in a special way using a secret key. After the
data is encrypted, it cannot be used unless decrypted using that key. Various
encryption algorithms exist in the industry, such as Data Encryption Standard (DES),
triple DES, and Advanced Encryption Standard (AES). One very important choice
made in encryption is the length of the key in bits. A very short key, for example 10
bits, has only 1,024 possible values (2^10), so it would be straightforward to attempt
decryption using all 1,024 possibilities, look at the results, and know which one was
correctin maybe a second of computer time on a PC. This is called cracking the
encryption. A very long key, for example 1,000 bits, has 2^1000 possibilities, which
are far too many to try in a million years using all the computers in existence.
However, encrypting using a 1,000-bit key is quite processor intensive and
completely impractical today at disk I/O speeds. Practically speaking, to run at disk
speeds an encryption algorithm and key length must be chosen for which a highspeed encryption/decryption VLSI device exists already. This means that if a wellfunded and expert government organization really wanted to crack what was on
that disk they could, but no commercial organization (and certainly no hacker) could
do so. Such a length today is between 100 and 150 bits. (Triple DES achieves what
is considered 112-bit strength by encrypting three times with separate 56-bit keys.)
UC434S F.00
12 -17
Keys must be generated randomly: if a key can be guessed, including guessed by

knowing a weakness of the random number generator it came from, security is
compromised.
Encrypting data on disk requires appropriate key management, which must balance
the difficulty of changing a key (all data must be rewritten) with the risk of loss of a
key (in which case all data is lost). Keeping more copies of a key makes it more
likely one will be compromised; keeping only one or a few copies makes it more
likely the key will be destroyed by a system failure or human error. Key management
software exists in the industry today, but success in managing encrypted data today
depends far more on people and processes than on technology.
Data can be encrypted on disk or tape today using the same box products used for
data-in-transit encryption. HP has considered encryption within storage controllers
such as disk arrays over the past decade; at this time the cost and complexity of such
a feature are not consistent with the needs of most of our customers. HP will continue
to consider this feature in future products based on cost, complexity, and evolving
customer needs.
Data on tape, optical disks, other media (nearline storage)

Backup tapes are more easily removed, not just from a secure data center but from
sometimes less secure offsite storage, and their loss is less easily detected than disks.
This creates the opportunity to encrypt data stored on backup tapes.
Such encryption is possible today, using the same encryption boxes previously
discussed. Key management becomes a very challenging issue since keys for every
known backup tape must be retained if the tape is to be retrievable and useful.
12 -18
UC434S F.00
SAN Security
Management security
Management security
Authentication of
administrators
Single sign on
Selective
administration
capability
Role-based access
Error tracking
Centralised
management view
Management Security
While far less exotic than encryption technology, basic management security for
storage devices is the most important area to focus on today, and is in transition.
Historically in the days of SCSIand today for the disks in a PCstorage is entirely
owned by a single system, any management software for that storage runs on that
system, and the only storage security (or storage management security) is what that
system provides.
As storage became shared by many systems, typically there was a management
utility installed on one or more of those systems, and the storage administrator was
required to supply a password to manage a particular array. This is normal practice
in the industry today, and works quite well when there is a single administrator of a
modest amount of storage.
However, as storage requirements have grown rapidly over the past few years and
pooling of free storage at a SAN level rather than inventorying free storage perapplication have become common, it has become much more important to have
multiple administrators, each with much more granular permissions as to which
actions they can perform, on which storage devices. Fortunately, this problem has
long been faced by the administrators of large numbers of servers, and technology
addressing it is well established. Single sign on is enabled by protocols such as
RADIUS (historically Remote Authentication Dial-in User Service) that forward a
logon request to a central server for validation, and by central servers such as
UC434S F.00
12 -19
Microsofts Active Directory or Netscape Directory Services (NDS) on various UNIX

platforms by way of the Lightweight Directory Access Protocol (LDAP), and others.
Creating a complete central directory environment for an entire organization is a
significant undertaking, beyond the scope of this paper. See Central directories and
challenge/response protocols background in the Authentication section.
Audit trails and logs must show which administrator performed a given action. This
becomes very difficult to administer on a per-device basis, and a number of
problems (even attacks) are visible only when such data is visible across all devices.
Tools to present a single, searchable view of logs are necessary in large installations.
Centralization of both authentication and authorization in the data center (if not the
enterprise) has been under way for some years. While in widespread use in larger
data centers and larger user populations today, this centralization is finding its way
into even modest organizations. Aspects of the data center like storage, which today
uses more traditional password schemes, are evolving to use centralized
authentication and authorization. For storage, this evolution started with tools able to
manage multiple devices of the same type, and is now moving toward toolsets with
single sign on that address a variety of storage or even a variety of servers and
storage together.
It is important to understand that this change is taking place product by product, in
the ordinary course of major releases, so as not to disrupt current installations and
current procedures. Small installations and customers who have not implemented a
central authentication server will still be able to use products the traditional way.
Authentication
Instead of having distinct (user id, password) logons for each system or device, a user
(application or person) has a single identity. Logging on to a given server, or to the
management port on a given device, appears to be by that device but in fact is
delegated to a data centers (preferably enterprises) central authentication server,
using RADIUS or a similar protocol. While common for user logons to systems today,
use of this technique to administer devices is just now starting to occur.
Technically, single sign on is usually accomplished by the users computer receiving
and holding a token (time limited key) as the result of logging on, which can be
transparently presented in response to future logon (authentication) requests. The
details of how such a token can be used without eavesdropping and impersonation
are very interesting technically but beyond the scope of this paper. RADIUS is a
protocol that, when user A is logging on to server B, allows B to ask the
RADIUS server is this logon valid? rather than maintain its own copy of the
user/password file.
Central authentication services offer several benefits. Single sign on gives a user only
one password to remember, so it is practical to change it periodically. If a user
leaves the organization, there is only one logon of concern, and it can be revoked
quickly and productively. The organization can move with relative ease from
12 -20
UC434S F.00
SAN Security
something you know (a password) to something you have and something you
know (a card and a PIN or password) authentication.
Authorization
Beyond single sign on, the second difference from current storage management
practice is that there will no longer be an administrator logon to manage a device,
which has all privileges and is shared by all administrators. Rather, administrator of
device x is a role that can be assigned to an individual.
The third and perhaps most important difference is that administrative privileges can
be granted in a fine-grain way. For example, one administrator could be given the
right to view anything but change nothing in a particular storage subsystem, while a
more senior administrator could make changes. Roles are predefined sets of
permissions that can be assigned to a particular person.
Again, the customer continues to have the flexibility to organize server and storage
administration. A small organization can continue to have a single administrator with
all permissions, while a larger organization might continue to have separate server,
storage, and network administration departments, each with varying permissions
based on specific individuals roles.
Audit
All configuration changes and other significant events should be logged, so that
problems of any sort (including security breaches) can be traced to their origin.
Understanding which administrator made the erroneous configuration change, and
when, makes it much easier to find and correct the process or procedure breakdown
that led to the error.
A centralized view of the audit trails/logs from the various devices in the data center
is important. The ability to query the collective set of logs rather than individual
elements is very important in tracking down issues, whether they are security
intrusions, administrator errors, or other problems. Overall security administration
may call for specific reports to be periodically generated from these logs.
When data is automatically moved between devices, such as in archiving,
Hierarchical Storage Management (HSM), or ILM, the software performing the data
movement must be duly authorized, and logs of such movement must be kept. This is
an emerging area; this is a goal rather than standard practice today.
UC434S F.00
12 -21
Local Authentication on a switch
The switch maintains the user name and password locally and stores the password
information in encrypted form. The users are authenticated based on the locally
stored information.
12 -22
UC434S F.00
SAN Security
Role-Based Access Control (RBAC) B-Series
Role-Based Action Control (RBAC) is the method that defines user accounts and
assigns a pre-defined set of permissions on the accounts based on those roles. This
allows a restricted list of commands that can be issued by that account, on a switch
or within the fabric. There can be up to 15 user defined accounts that can be
created, and the default accounts User and Admin may be disabled.
WARNING: Before disabling the Admin account, make sure that a user defined
account with the admin role has been defined and works, if the Admin account is
disabled without an account with admin privileges being enabled it will mean that
the switch can not be fully managed.
All the following commands require the Admin Role:
userConfig show -a shows all account information for a logical switch
userConfig show username this command shows account information for a specified
user account
To create an account:
Userconfig --add name r role eg: userconfig add battleaxe r admin
To delete an account:
userConfig delete username eg: userConfig delete battleaxe
UC434S F.00
12 -23
To change roles:
userconfig --change username [-r rolename] eg: userconfig --change battleaxe r
SecurityAdmin
Administrative Domains B-Series

Admin Domain (AD) is a logical grouping of fabric elements that define switches,
ports and devices that can be viewed, managed and modified. An Admin Domain
is a filtered administrative view of the fabric, which permits access to a configured set
of users. By using AD, the fabric can be partitioned into logical groups and
administration control of these groups can be allocated to different users accounts so
that theses accounts can only make changes to the switches, and ports assigned to
them and not the rest of the fabric. Dont confuse zones with admin Domains, zones
define which host and storage device can communicate with each other, where
admin domains, define which users can mange which switch, host and storage
device.
There are up to 256 AD in a fabric, which 254 are user defined and two are system
defined, the Admin Domains are number 0 255 and designated by Name and
number, if only the number is specified AD is automatically is assigned as the name
to the number. The Admin Domain administrator can define up to 254 ADs (AD1
AD254) in the AD database; it is recommended however that no more than 16 active
Admin Domains run concurrently. It has been found that having more than 16 active
Admin Domains can cause performance degradation and unpredictable system
behavior.
Dont confuse a Admin Domain number with the Domain ID of a switch, as the AD
identifiers have a range of 0 -255 and define group of switches, host and devices
that can be managed, whilst the Domain ID range is 1 239 and uniquely identify
the switch in the fabric.
Admin Domains offer a hierarchy of administrative access. To manage Admin
Domains, a physical fabric administrator account is required. A physical fabric
administrator is a user with the Admin role and access to all Admin Domains (AD0
through AD255). Other administrative access is determined by defined RBAC role
and AD membership. The role determines the access level and permission required to
perform an operation.
Creating a new user account for managing Admin Domains

1.
Connect to the switch and log in as admin.
2.
userconfig --add username -r role -h home_AD -a "AD_list"

eg: userconfig --add ad1admin -r admin -h hp_ad1 -a "hp_ad1"
12 -24
UC434S F.00
SAN Security
The following example creates new user account ad2admin with an admin role,
access to Admin
Domains 1 and 2, and home Admin Domain set to 2.
sw5:admin> userconfig --add ad2admin -r admin -h 2 -a "1,2"
By creating defined user accounts assists in the tracking of changes within the san
environment, tracking who did what and when, to enable tracking enter the following
command:
TrackChangeSet 1
Users can also be created using the Switch admini gui. both methods are
covered in the lab after this module.
UC434S F.00
12 -25
Role-Based Access Control (RBAC) C-Series
Role-based access assigns roles or groups to users and limits access to the switch.
Access is assigned based on the permission level associated with each user ID. Your
administrator can provide complete access to each user or restrict access to specific
read and write levels for each command. This can also be on a per-VSAN basis.
SNMP and CLI access rights are organized by roles. Each role is similar to a group.
Each group of users has a specific role, and the access for that group can be
enabled or disabled.
By default, two roles exist in all switches:
_ Network operator (network-operator): Has permission to view the configuration only. The
operator cannot make any configuration changes.
_ Network administrator (network-admin): Has permission to execute all commands and
make configuration changes. The administrator can also create and customize up to 64
additional roles.
The two default roles cannot be changed or deleted.
Switch access security

Each switch can be accessed through the CLI or SNMP.
Secure switch access: Available when you explicitly enable Secure Shell
(SSH) access to the switch. SSH access provides additional controlled security by
encrypting data, user IDs, and passwords. By default, Telnet access is enabled on
each switch.
SNMP access: SNMPv3 provides built-in security for secure user authentication and
data encryption.
12 -26
UC434S F.00
SAN Security
RADIUS Authentication
RADIUS (Remote Authentication Dial-In User Service) is a protocol for carrying all
authentication, authorization, and accounting (AAA) information between devices.
When configured to use RADIUS, the switch acts as a network access server (NAS)
and a RADIUS client. The switch sends all authentication, authorization, and
accounting (AAA) service requests to the RADIUS server. The RADIUS server receives
the request, validates the request, and sends its response back to the switch.
The supported management access channels that integrate with RADIUS or include
serial port, Telnet, SSH, Web Tools, and API. All these require the switch IP address
or name to connect. The RADIUS server accepts both IPv4 and IPv6 address formats.
Note: RADIUS protocol only encryptspasswords
Brocade also support LDAP lightweight directory access protocol (LDAP) using
Microsoft Active Directory in Windows at the same time. A switch can be
configured to try both RADIUS or LDAP and local switch authentication.
Cisco also supports TACACS+ which is a client-server protocol this uses TCP (TCP
port 49) for transport requirements.
The addition of TACACS+ support in enables the following advantages over
RADIUS authentication:
Provides independent, modular AAA facilities, authorization can be done without
authentication. It performs independent of servers if it is configured to use a local
database. Can utilize TCP to send data between the AAA client and server,
providing reliable, connection oriented sessions and encrypts the entire protocol
UC434S F.00
12 -27
payload between the switch and the AAA server to ensure higher data
confidentiality..
12 -28
UC434S F.00
SAN Security
SAN security practices

SAN security practices
HP StorageWorks SAN hardware and software components incorporate features that

can be used to implement a secure data storage system. The following table shows
the appropriate use of these security features in various environments. The enterprise
storage system environment is a typical mid-sized to large IT installation used in a
business. The service provider storage system environment is a large installation in
which several customers share a single IT infrastructure.
UC434S F.00
12 -29
Planning SAN security prevention

Planning SAN security prevention
Planning for the following is essential
Unauthorized and/or unauthenticated SAN access
WWN spoofing
Management controls allowed from different access points
Multilevel password controls
Management ACLs and encryption of passwords in certain
interfaces
Port-level ACLs
Public key infrastructure based authentication and security
Unauthorized and/or unauthenticated SAN access can be because of:
Insecure management access
World-Wide Name (WWN) spoofing
Management controls allowed from different access points
12 -30
Multilevel password controls to prevent unauthorized and unauthenticated SAN

access
Management ACLs and encryption of passwords in certain interfaces
Port-level ACLs
Public Key Infrastructure based authentication and security Digital certificates
UC434S F.00
SAN Security
Data path and management path security in

practice
HP divides the responsibility for SAN security into two parts:
Data Security refers to the protection of the communication path used to move
user data through the SAN.
SAN Management Path Security refers to the protection of the communication
path used to move management information through the SAN.
This is a functional distinction. In some cases, the same physical connection is used
for both user data and management information.
The HP storage security model is implemented as three distinct areas.
UC434S F.00
The overall security of the storage system is an integral part of the total solution
security and is deployed within the context of a comprehensive understanding of
the system, developed and delivered by HP Professional Services.
The software components of the storage system provide management path
security by controlling operator access rights and securing the SAN
management communication paths.
The hardware components of the SAN provide data path security by controlling
storage array access and governing the SAN fabric configuration control
mechanisms.
12 -31
Storage security in an enterprise environment

Storage security in an enterprise
environment
Security expectations
SAN component security attributes
Response to attack
Use good employment practices to minimize
Checklist
malicious attacks.
Provide computer system security awareness training
for all personnel.
Perform routine user account management at the
server.
Enable disk quotas for all users.
Locate storage systems and Fibre Channel switches
in a secure area.
Enable passwords on all switch configuration ports.
Enable Selective Storage Presentation for all logical
units.
Disable SES management interface to Fibre Channel
switches.
Perform routine periodic security audits.
In a business enterprise, computer systems can be shared between two or more

departments. The systems are managed and operated by information systems (IS)
organization, which has enterprise-wide responsibility for the computing environment.
All the people in the enterprise work toward a common business goal, but the day-today interests of the departments can vary widely, depending on the business climate,
time of year, or product development issues. Each department has specific computing
requirements that must be met by the IS organization.
There can be wide differences in the need for data security. For example, a typical
accounting department has strict security guidelines, while the marketing department
might be willing to tolerate more risk.
The IS organization can try to achieve efficiency by placing the computer equipment
in a single central location. A considerable amount of computer and storage
hardware is required for an enterprise of moderate size. This discussion assumes that
the storage for all the departments is located in a single SAN storage system. Servers
are distributed throughout the facility.
The IS organization must implement a computing system that meets the security and
capacity requirements of all the departments to which it provides service, and the IS
security manager must implement a security plan that is suitable for the needs of the
enterprise.
To meet the security requirements, many security managers specify a centralized
machine room located in a secure area. This method substantially reduces the
12 -32
UC434S F.00
SAN Security
security risk for the storage system because the ordinary users of the system do not
have physical access to the machines.
Security expectations
This environment has a requirement for a high level of storage system security.
Protection is needed against unauthorized, accidental, and malicious data access
attempts. The required security level is set by the department with the most strict
security needs.
SAN component security attributes

The following features are used to provide security in this environment.
Traditional user account security is in effect in the servers. This configuration protects
each user account against accidental access by an unauthorized user. Disk quotas
are enabled for each account, which prevents a user from consuming all of the
storage capacity allocated to the server.
The HBAs pass user I/O requests to a Fibre Channel switch. Communication is done
using Fibre Channel fiber optic cables. These cables pass from the servers into the
secure area that holds the storage systems.
The SAN switches are shared by all the users and servers in all departments in the
system and are located in the secure area. Configuration management of the
switches is done by the system manager using the web management interface. The
interface is protected by password to prevent unauthorized changes to the switch
configuration.
Data is stored in several HP StorageWorks storage systems. Access to each logical
unit is controlled by the Selective Storage Presentation (SSP) feature of the array
controller.
Response to attacks
Two attack scenarios are possible in this situation. Accidental inappropriate data
access requests might be made by any user, and malicious attempts to make an
inappropriate data access requests might be made by a user.
Inappropriate read and write requests by system users are routinely handled by the
operating system. Disk mounting requires a privileged account, and directories are
protected by ACLs. The benign server environment puts little stress on the security
capabilities of the storage system.
Because the storage systems are located in a secure area, the risk of inappropriate
access to the array controllers is limited. There is some risk that the fiber optic cables
might be tapped, but this requires a technical approach that is unlikely in this
scenario.
UC434S F.00
12 -33
Denial of service attacks initiated by system users, whether accidental or purposeful,

is not fully protected against. A user could write a program that issues a useless but
very high I/O load and consume most of the I/O operation capability of the storage
system.
Checklist
For a SAN storage system that requires a moderate level of security and where the
storage systems and Fibre Channel switches are located in a secure area, the
following steps are required.
12 -34
Use good employment practices to minimize malicious attacks.
Provide computer system security awareness training for all personnel.
Perform routine user account management at the server.
Enable disk quotas for all users.
Locate storage systems and Fibre Channel switches in a secure area.
Enable passwords on all switch configuration ports.
Enable Selective Storage Presentation for all logical units.
Disable SES management interface to Fibre Channel switches.
Perform routine periodic security audits.
UC434S F.00
SAN Security
N_Port authentication
E_Port authentication
Encryption at rest and on the move
Zoning
LUN Masking
Virtual fabrics traffic separation
Role-based Access Control
UC434S F.00
12 -35
Authentication
Authentication
Authenticate port wwn of newly
connected device against ACL in
fabric
Authenticate switch wwn before
E_Port approval
Specifies device participation in

fabric
May lock wwn against physical

port
1. FLOGI
2. Check ACL
3. Allow/deny
Distributed
ACL
Helps prevent wwn spoofing
May involve Key exchange:

DH-CHAP - a secure key-exchange authentication
protocol that supports both switch-to-switch and host-toswitch authentication.
Specifies which devices can participate in a fabric and locks them down to a specific
port within the fabric to prevent the addition of a device to an unauthorized port.
Organizations can use this policy as a WWN spoofing countermeasure by
preventing a device that is configured to mimic an existing device from joining a
fabric unless the device being spoofed is first disconnected then physically replaced
with an unauthorized device.
DHCHAP is a mandatory password-based, key-exchange authentication protocol that
supports both switch-to-switch and host-to-switch authentication. DHCHAP negotiates
hash algorithms and DH groups before performing authentication. It supports MD5
and SHA-1 algorithm-based authentication.
12 -36
UC434S F.00
SAN Security
Today it seems to be a regular occurrence that, a laptop, a memory key, a DVD have
gone missing in the post or left on a train or coffee bar. With encryption although
embarrassing it does eliminate all the effects of the information loss in restricting
access to data by unauthorized persons. In many countries laws are in place that
require immediate notification for breach of security were personnel details may be
involved, but are exempted if the data was encrypted.
How a company or government agency is perceived after a data loss can have a
major impact on public and business perception of that organization. This could
lead to loss of orders, devaluation of stock or mistrust of organizations, government
and public in keeping individual information safe and secure. In todays hi-tech
environments encryption of data is critical ensuring the integrity of the organization
and data enshrined with in it..
UC434S F.00
12 -37
FCIP encryption and Data encryption at rest

FCIP encryption and Data encryption at rest
Standards-based IPSec protocol to secure FCIP and iSCSI
traffic
Hardware-based end-to-end authentication and encryption
B-Series
Supported on:
HP StorageWorks 400 MultiProtocol Router , MP Router

Blade (FR4-18i)
C-Series
Encryption Data at rest

Brocade Encryption Switch
Brocade Encryption FS8-18 Blade
Supported on:
MDS 9216i, MPS 14+2

module
As discussed previously IPSec is supported on C-Series and B-Series, which can have
a performance, impact on applications due to the overhead of encryption. Each can
benefit from being deployed and managed as part of a computer network. This
leads to benefits that include high availability, scalable performance with low
latency, and simplified load balancing through network traffic management. To meet
the security concerns and government compliance Cisco and Brocade have
developed their own encryption solutions. Both companies have created solutions
that simplify deployment and also increase performance of the encryption process
within the San.

The installation and configuration of the C-Series SME is aimed at being simple and
also non-disruptive, as the encryption engines are fitted on to modules that slide into
a MDS 9500 or MDS9200. This eliminates the requirement for rewiring of the SAN
configuration, and also removing the need to purchase and manage extra switch
ports, cables and appliances.
Key benefits of the SME is that it can be utilized to encrypt any traffic from any
VSAN, by using the CLI or Cisco Fabric Manager no extra software is required to
manage the encryption so in effect reducing operating expenses.
12 -38
UC434S F.00
SAN Security
Brocade Encryption Switch / Encryption FS8-18 Blade

On September 22nd 2008 Brocade introduced Brocade Encryption Switch and the
Brocade FS8-18 Encryption Blade which is for use with the Brocade Director class
DCX Backbone. The Brocade Encryption Switch is a full-feature 32-port 8 Gbit/sec
Fibre Channel switch and the Brocade FS8-18 Encryption Blade for the Brocade DCX
Backbone is a 16-port 8 Gbit/sec blade each provides up to 96 Gbit/sec of
encryption processing power. Both products are managed either by the Cli
(Command Line Interface) or through the Data Center Fabric Manager (DCFM)
application software.
Both Cisco and Brocade provide encryption solutions that address, government
regulations, and security concerns on safeguarding data at rest. The encryption
solution is transparent and encrypts the data inside the SAN environment, without
impacting on business applications.
UC434S F.00
12 -39
12 -40
UC434S F.00
SAN Security
Lab activity
Lab
activity
Module 12, Lab 1 Fabric
Security
28
UC434S F.00
uc434s c.01
12 -41
12 -42
UC434S F.00
Data protection
Module 13
Objectives
Objectives
Discuss backup, types and their differences
De-duplication - Accelerated de-duplication & Dynamic
de-duplication (hash-based chunking)
Distinguish between synchronous and Asynchronous
replication
Explain split mirror and snapshot replication in more
detail
UC434S F.00
13 -1
Data Protection
Data Protection
Maintaining availability of data is the primary goal
of data protection
Why is it important?
50% companies which loose data go out of business
immediately
90% do not survive more than 2 years following that
loss
Figures are based on research from Baroudi Bloor International; Sarbanes-Oxley

Compliance Journal, December 2005.
50% companies which loose data go out of business immediately
90% do not survive more than 2 years following that loss
If Raid is designed to protect data against bit and byte errors, then the concept of
business continuance is to handle the perceived problems at the server and
application levels. This maybe achieved by making separate copies of data, this can
be performed in ways that allow the original data to still be accessed by users, while
the second copy is being created. This copy maybe created locally, or a copy may
be stored remotely which is routinely synchronized with the original data to ensure
integrity between the original and its copy. This copy process may be performed
through the operating system, application or storage system solution, such as Hp
Storage Works Business Copy.
13 -2
UC434S F.00
Data protection
Challenges in Data Protection

Challenges in Data Protection
Backup
Window
Inconsistent recovery
Recovery time too long
Impact on production applications
Protection gaps
Disaster recovery
Compliance
Backup Window - The time it takes for a backup job to complete. This may be more
time than is available.
Inconsistent Recovery Data backed up to tape during a backup job may not be
integral when/if restore is attempted. Backup has not been verified.
Recovery time too long Unacceptable delay before restoration may occur,
particularly if required data are situated on tape in offsite store.
Impact on production applications application downtime associated with regular
backup job may not suit business model
Protection gaps failed or inconsistent backups may result in protection gaps
Disaster Recovery DR planning may be incomplete or unworkable test your plan!
Compliance there may be a regulatory requirement for some types of data to be
retained in a specific way for a finite duration.
UC434S F.00
13 -3
Not all data are equal

Not all data are equal
Categorizing data within the business is a key first step in
enabling prioritization during the backup AND restore
processes
Mission Critical
Important
Archived
Current data
Required 24x7
Catastrophic loss could cause failure of
business
Recent data
Not needed 24x7
Loss may cause business disruption
Historical data
Accessed infrequently
Often required by law
Within any organization there is a stratification of data importance. Understanding

this stratification will enable IT managers to allocate appropriate time and resources
in a proportional manner.
13 -4
UC434S F.00
Data protection
Recovery operations
Recovery operations
Recovery Point Objective
Maximum tolerated time prior
to disaster over which data
may be lost as a
consequence of recovery
Recovery Point
Years Days Hours Mins Secs
Recovery Time Objective

Maximum tolerated time
required to bring
applications and data back
to operational state
Recovery Time
Secs Mins Hours Days
Time
Recovery Point Objective

Maximum tolerated time prior to disaster over which data may be lost as a
consequence of recovery.
Recovery Time Objective

Maximum tolerated time required to bring applications and data back to operational
state.
UC434S F.00
13 -5
Protection and recovery methods

Protection and recovery methods
Protection
Methods
Tape
backups
Disk
Real time
backups
replication
Snapshots
Archives
Recovery Point
Years Days Hours Mins Secs
Recovery
Methods
Instant
recovery
Roll back
Tape
restores
Recovery Time
Secs Mins Hours Days
Time
Protection methods starting with the longest protection time:
Tape backups and Archives
Disk backups and Snapshots
Real time replication
Recovery methods:
Instant recovery
Roll back
Tape restores
Disk Restores
Restore from disk is less time consuming than restoring from tape, especially if data
on tape has been encrypted, compressed, or de-duplicated.
13 -6
UC434S F.00
Data protection
Data Protection Technologies

Data Protection Technologies
Physical
Tape
Virtual Tape
Replication
Local
Remote
Clustering
Physical tape will always be the foundation of a robust data protection strategy
offering low cost/GB storage and off-site vaulting capabilities. However, in many
cases an increase in backup performance can be achieved by using disk-assisted
backup techniques1.
Disk has the advantage of being random access and does not suffer the same
performance (repositioning) issues when backing up lots of small files. Additionally
backup to disk does not generally suffer from some of the error conditions that can
cause backup jobs to tape to fail, for example, no media in the media pool, media
coming to the end of its useful life, tape jams, and robotic failures.
Tape-based data protection

Magnetic tape has been used to provide data protection for more than 50 years
and it is still the most cost-effective technology for high-capacity and long-term data
protection. That is because tape has a number of advantages that have yet to be
eclipsed by other technologies:
Tape media is small for its capacity as well as transportable; it can therefore be
stored offsite easily.
Its long shelf life of up to 30 years makes it a dependable medium for archiving.
Because tape is removable, its capacity is effectively infinitely scalable.
UC434S F.00
Library solutions can be integrated easily into many environments to provide

automated, multiplecartridge backup sessions.
13 -7
Disk-to-disk (D2D) backup

In most disk-to-disk (D2D) solutions, a disk array is divided into backup targets for
each host server being backed up. Once a host has written to the disk, the migration
to physical tape occurs through that host. This means that the host is involved both in
the first backup to the disk and in the transfer to physical tape.
Virtual tape
Virtual tape* is a disk-based storage device that appears to the LAN or SAN as a
tape drive, tape autoloader, or tape library. By presenting a virtual tape device to the
LAN or SAN, the pool of storage within it may be shared dynamically among
multiple hosts. Virtual tape can improve backup and restore performance
dramatically because virtual tapes are easy to provision.
Replication
Snapshots, clones, and mirrors* allow backups to be performed with no interruption
to your applications. They also allow data to be restored instantly from saved images
on the disk array.
Clustering
Clustering provides protection against basic hardware failure. A cluster of servers
provides fault tolerance because if one server fails in a system, one or more
additional servers are still available to take over operations.
13 -8
UC434S F.00
Data protection
Direct backup tape

Direct backup - tape
Backup
application
LAN
Management
SCSI
connection
= Primary storage
= Tape drive
The preceding diagram shows a basic server backup environment (also referred to as
a local backup) in which each server connects to its own backup device through a
SCSI bus. The operator loads a dedicated instance of the backup software for each
server that has a backup requirement. The backup software reads the data from
primary storage then writes the data to the backup device.
The operator controls the application locally or remotely, depending on the remote
management capabilities of the application. The storage media for each server is
managed locally and manually.
In this arrangement, the speed of the backup device affects backup performance.
Backup data and network traffic each travel on separate lines. However, one
advantage of this backup method is that backups do not consume LAN bandwidth.
Basic server backup advantages and disadvantages
UC434S F.00
Advantages
Disadvantages
Fast
Does not consume LAN bandwidth
Relatively expensive
Must manage each server individually
13 -9
Centralized server backup

Centralized server backup
LAN
Management
Backup
application
SCSI
connection
Client
agent
software
= Primary storage
= Tape drive
With the introduction of client push agents, backup devices no longer require
attachment directly to the server in need of a backup; they can be located on a
different server attached to the LAN. The backup application runs on the server
hosting the backup devices, and client agents push the data over the LAN to the
server running the backup application. Media management difficulties decrease with
the consolidation into one backup device.
However, for installations with many servers, the LAN becomes the performance
bottleneck for backup. The additional traffic on the LAN consumes bandwidth that
could otherwise be used for business productivity. This backup traffic places new
constraints on the network when backups are performed, and the scheduling of
server backup windows becomes critical to the smooth operation of the business.
The following table displays maximum and typical LAN speeds that can be regarded
as the upper limits to backup data transfer rates over a LAN.
LAN type
Maximum
speed
Typical speed
10Base-T
100Base-T
FDDI
3.6GB/hr
36GB/hr
Similar to
100Base-T
2GB/hr
15 20GB/hr
Similar to
100Base-T
Fibre Channel
13 -10
360GB/hr
280GB/hr
UC434S F.00
Data protection
Automated centralized backup

Automated centralized backup
LAN
Management
Backup
application
Client
agent
software
Tape library
= Primary storage
Using a LTO tape library adds capacity and automation to further reduce the media
management problems.
Example
With all data flowing through one server, the backup speed is limited by the:
Capabilities of the LAN
I/O capabilities of the server
Because backups require most of the network bandwidth, they must be scheduled
during off-peak hours or during scheduled outage windows.
Automated centralized backup advantages and disadvantages
UC434S F.00
Advantages
Disadvantages
Centralized management
Tape automation
High speeds from backup server to tape device
LAN bandwidth consumption

Single point of failure (backup server)
13 -11
Centralized SAN backup

Centralized SAN backup
LAN
Management
Backup
application
Client
agent
software
SAN attached tape library

= Primary storage
In this solution the San is used to move data required for backup, and is sometimes
defined as a Lan Free backup. Although in reality the lan is still used to send bytes of
information regarding the progress of the backup and which files have been
processed.
The process may be something like this:
a.
Backup server issues a backup command via the lan to the backup client.
b.
The request is read by the client, regarding the data and type to be backed
up via the San.
c.
Data is the written from the backup client to a tape library over the san.
d.
As each file is written to the tape library, the client also issues a few bytes of
control data to the backup server these commands however travel over the
lan.
e.
When all the data has been successfully transferred to tape, the lan is used
again by the backup client to communicate with the backup server
indicating completion of the backup.
Centralized server backup advantages and disadvantages
13 -12
Advantages
Disadvantages
Centralized management
LAN bandwidth consumption

Single point of failure (backup server)
UC434S F.00
Data protection
Tape libraries
Tape libraries
High
performance and
reliability
Sophisticated robotics to
automate tape-changing and
backups
Allow SAN-based centralization
and reduced management costs
LTO-4 compatible
1.6TB compressed per tape
240MB/s compressed data
transfer
AES 256 bit hardware encryption
Each of the HP StorageWorks tape libraries offers high performance for reliability.
The libraries commonly used in SANs are the HP StorageWorks ESL E-Series .
Note: Refer to the Designing and Implementing the HP StorageWorks Enterprise
Backup Solutions WBT course for more detailed information on the HP tape libraries.
All the libraries employ sophisticated robotics to automate tape-changing functions
and enable backups of thousands of gigabytes of data. The HP library mechanisms
place and remove tape cartridges with minimum contact to surfaces through a
precision-grip cartridge handling system that emulates the human hand.
UC434S F.00
13 -13
Zoning for backup

Zoning for backup
Why
zone?
Reduced stress to tape drives from polling agents

Fewer unnecessary discoveries by Network Storage
Router or interface controller
Reduced potential for target and LUN shifting, which
could affect backup software configuration
Recommendations
Small fabricUse the lowest port on SAN switch for

tape controller
Small to medium fabricUse host-centric zoning
Large fabricUse host-centric zoning; split tape and
disk targets onto different zones
Zoning provides the following benefits in an EBS configuration:
Reduced stress to tape drives from polling agents

Fewer unnecessary discoveries by the Network Storage Router or the interface
controller
Reduced potential for target and LUN shifting, which could affect backup
application configuration
Zoning may not always be required for configurations that are already small or
simple. Typically the bigger the SAN is, the more zoning is needed. HP recommends
the following for determining how and when to use zoning.
13 -14
Small fabric (16 ports or less)May not need zoning, depending on the type of
hosts and storage devices. If no zoning is used, it is recommended that the tape
controllers reside in the lowest ports of the switch.
Small to medium fabric (16 - 128 ports)Use host-centric zoning. Host-centric
zoning is implemented by creating a specific zone for each server or host, and
adding only those storage elements to be utilized by that host. Host-centric
zoning prevents a server from detecting any other devices on the SAN or
including other servers, and it simplifies the device discovery process.
UC434S F.00
Data protection
UC434S F.00
Large fabric (128 ports or more)Use host-centric zoning and split disk and tape
targets. Splitting disk and tape targets from being in the same zone together will
help to keep the tape controllers free from discovering disk controllers, which is
unnecessary unless extended copy is required.
To implement a host-centric zone, create a specific zone for each server or host
and add only those storage elements to be used by that host. This configuration
prevents a server from detecting any other devices on the SAN (including other
servers) and simplifies the device discovery process.
13 -15
Backup performance considerations

Backup performance considerations
Feed source
I/O capability
Application
Filesystem
CPU and RAM
Storage connections
File block size
File (data) compression ratio (hardware/software
compression)
Tape technology
HP developed high-performance EBS to meet the demand for reliable secondary

storage. For these backup solutions to function with the performance their critical role
demands, of their implementation must optimize the following:
Configuration of the primary storage system
Type of connection to the tape storage system
Type and configuration of the tape drives
Compression ratio and data verification
Block sizes and formatting of tapes
Configuration of primary storage

The configuration of primary storage impacts the performance of secondary storage.
The transfer rates and type of hard drives used in the primary storage solution help
determine how quickly data can be moved from the drives to the secondary storage
system. The faster the transfer rate of the hard drive, the faster the data can be
accessed.
The number of spindles in the primary storage system also affects secondary storage
performance.
The controller for the primary storage determines how quickly data can be retrieved
from the system.
13 -16
UC434S F.00
Data protection
Performance
To analyze speed and performance, the entire backup process must be examined as
a system of components. The backup process can be divided into a set of five
components that affect performance. Each of these components must be thoroughly
understood and factored in to the backup equation to determine the maximum
performance in any specific situation.
The five components of the EBS are:
Feed source - This is usually the hard disk primary storage system, but it can be
network-connected storage or even a remote system.
Storage connection for EBS - This is a Fibre Channel connection.
File block size - EBS supports up to a 32KB transfer block size for NetWare and
a 64KB transfer block size for Windows NT 4.0 or Windows 2000.
File (data) compression ratio - The amount of compression has a direct impact on
the rate at which a DLT tape drive can read and write data.
Tape drive (secondary storage) system - For the EBS, these systems are HP
StorageWorks libraries.
Feed source, primary storage, and controller type

The type of controller that is used has a direct effect on the speed at which the server
can send data to the tape device. HP tests show that it is necessary to read from the
primary storage device at a speed at least three times the backup rate (3:1) of each
tape drive in order to stream data at optimal speeds. Therefore, if the controller
cannot feed data to the tape drive at a fast enough rate, the drive performance slows
down because of idle time on the tape drive.
For slower hosts, LTO tape drives can adapt the tape speed in order to match the
feed data rate to maintain streaming of the tape. However, the tape drive would not
be writing at its maximum possible speed if it must slow down.
UC434S F.00
13 -17
Virtual Tape Libraries

Virtual Tape Libraries
Emulates
popular libraries and

tapes to backup software
Do more backup jobs in parallel
Reduced backup times
Fast single file restore
Compression uses disk capacity
efficiently
Fibre Channel and iSCSI
interfaces
Configured to use RAID6
A Virtual Tape Library (VTL) is a dedicated computing appliance that emulates the
drives of a physical tape library and stores backup images to disk. Backup
applications, like HP Data Protector, use the VTL emulated tape and library devices
for backups when in fact it is an array-based appliance. The VTL consists of three
components: computer hardware, a RAID-based array of disk drives, and application
software which emulates a tape library.
Improved backup and restore efficiency and Performance

The VLS can emulate multiple tape drives and tape devices, allowing for
simultaneous backup of multiple servers to a single device.
Accessing files and data from disk-based storage simplifies the restore process and
can improve performance from hours to minutes.
Improved data availability

The VLS emulates enough devices that you are now able to service restore requests
during the backup window something that is cost- prohibitive with a strictly
physical tape solution.
Unattended (automated) backup

HP Virtual Library Systems work with compatible backup software applications to
provide automated backup of all application and file data while simplifying and
improving the reliability of the backup process.
13 -18
UC434S F.00
Data protection
Reduced media management requirements and costs

The VLS uses disk-based storage to store all application and file data, decreasing
both the issues and costs associated with tape media management.
Efficient use of storage capacity

The HP Virtual Library System includes user-enabled compression allowing the
effective capacity to double.
UC434S F.00
13 -19
VTL in practice
VTL in practice
Faster
Backups
Faster Restores
C onfig uration
P hys ical T ape
V irtual Tape (no compres s ion)
V irtual Tape (with compres s ion)
T ime
7
min 23 sec (443
s ec)
3
min 10 sec (190
s4ec)
min 12 sec (252
s ec)
MB /s
29.34
68.42
51.58
R es tore T ype
S erver 1 from phys ical tape
S erver 3
phys ical
tape
1 from virtual
tape
(no
ion) virtual tape (no
Scompress
erver 3 from
compress ion)
R es
tore
2
min
6 s Tecime
(126
s ec)
3 min (180 sec)
22 s ec
20 s ec
The example above shows five servers. Servers 1 and 2 have network connections to
the backup server, while Servers 3, 4, and 5 have SAN-attached storage residing on
the HP StorageWorks Modular Smart Array 1000 (MSA1000). The configuration
simulates a small mixed (LAN and SAN) backup environment. Approximately 2.6 GB
of known data (file sizes 64 K to 64 MB, 2:1 compressible data) is created on
Servers 1 and 2 (on the local drive) and on Servers 3, 4, and 5 as mapped LUNs on
the MSA1000 disk array.
The five servers are backed up to three different configurations:
A physical tape library with two physical LTO-2 drivesthe HP StorageWorks

MSL6030
A five-drive virtual tape library with no compression on the virtual tape drives
A five-drive virtual tape library with compression on the virtual tape drives
A total of approximately 13 GB is backed up and the time is measured as reported

by the backup application from start to final completion message.
Faster Backups
With configuration 1, the backup job must wait for physical tape drives to become
available for different servers (physical limit of two drives), whereas with the virtual
tape backups, a dedicated virtual tape drive is allocated to each server, so the
backup effectively happens in parallel.
13 -20
UC434S F.00
Data protection
Data compression within the VLS6000 series is performed in software in the VLS
node, and this increases the capacity of the VLS6105 in this example from 2.5 TB up
to 5.0 TB (with 2:1 compressible data). The downside is that because the data
compression is performed in software, the throughput is reduced, as can be seen in
the results.
Faster single file restores

The next test was to evaluate the restore times from physical media (already loaded
in the library) and from virtual tape. The physical media recovery time obviously
would be longer if it had to be retrieved from offsite. The restore consisted of the
same single file from approximately 50 percent into the directory structure of the
backup.
Time is measured as reported by the backup application start to finish.
UC434S F.00
13 -21
Disk to Tape
Disk to Tape
For a reliable and robust Data Protection policy HP
recommends using the backup application to migrate data
from Disk to Tape
HP strongly recommends the use of the backup application to migrate data from the
Virtual Library System to physical tape because using the backup application means
all the media (virtual and physical) is tracked in the backup application catalog. This
ensures reliable, robust data recovery. Tape technology has long been the standard
for protecting business data. Its portable, it has a long shelf life, its cost-effective
and it can hold a lot of data. And for these reasons, its still the best choice for
certain small businesses, depending on their recovery needs. But for other small and
medium businesses, or SMBs, a tape-based approach alone may not meet their
demand for business continuity. Thats why the best backup and restore plans today
dont use just tapethey incorporate disk, too. This two-tiered approach might be the
only way to get the specific data SMBs need back fast enough to keep their
businesses up and runningand now its actually an affordable strategy.
13 -22
UC434S F.00
Data protection
Data replication
Data replication
Split
mirror
Snapshots
De-duplication
Array-based replication
Host based replication (software)
Split mirror
Snapshots
De-duplication
Array-based replication
Host based replication (software)
UC434S F.00
13 -23
Split-mirror backup concept

Split-mirror backup concept
XP &
EVA*
Application host
M
0
M
1
M
2
Backup host
P primary LDEV
M mirror copy (MU0-2)
* - EVA MirrorClone supports one Mirror only
The general idea behind split-mirror backups is to stream the backup from the mirror
instead of the production disk. The mirror is typically connected to a separate host
(called the backup host) with a tape device attached. Usually, hardware mirror
technologies such as HP StorageWorks Business Copy XP or HP StorageWorks
Continuous Access XP are used to create the mirror.
Before a backup of a mirror can be started, a valid point-in-time disk image must be
created. The disk image must be consistent so that it can be fully restored. The mirror
must be established before proceeding with the backup. To create the backup image,
the mirror is split off the production disk at backup time.
Because the application host and backup host are different, all cached information
(database and file system cache) on the host is flushed to the disk before the mirror is
split off. Depending on the type of data to back up, flush the cache by:
Putting the database into backup mode
Taking the database offline
The split-mirror backup completes successfully with the file system mounted. However,
a successful restore of all files and directories cannot be guaranteed because cached
data is not written to disk before the split. HP therefore recommends dismounting a
file system before performing a spit-mirror backup.
13 -24
UC434S F.00
Data protection
If a database is running on a file system, there is no need to dismount the file system
because the database controls the write to the disk and ensures that data is written to
the disk and not to the file system cache.
For the online database backup, the backup image alone cannot be restored. The
archive log files from the application host are also needed. The archive log backup
can be started when the database is taken out of backup mode, which occurs right
after the mirrors are successfully split off their productive disks. This is true for Oracle,
SQL and certain other databases, but not necessarily true for all database
applications.
Mirror rotation
Mirror rotation relies on the ability of Business Copy to maintain up to three
independent secondary volumes (S-Vols) of one primary volume (P-Vol). The different
S-Vols are labeled as mirror units (MU#0, MU#1, and MU#2).
Data Protector can perform split-mirror backups of each of these mirrors. Users can
either supply one dedicated S-Vol or multiple S-Vols for backup. If two or more
mirrors are available, Data Protector automatically uses them in a cyclic fashion. At
the end of the backup, the S-Vol used is left split off the P-Vol, thus keeping the
backup versions on the S-Vol available for IR. For the next backup, another S-Vol is
used. This process provides a high level of data protection.
N.B. the number of MirrorClones for EVA may be less than the number available to
configure for the XP Array.
UC434S F.00
13 -25
Snapshot backup concept

Snapshot backup concept
EVA
& XP
Application host
S
Backup host
P primary LUN
S snapshot/child
The snapshot backup concept is similar to the split-mirror backup. The snapshot
backup is currently supported with the HP Virtual Enterprise Array Systems XP Storage
Systems.
Snapshots can be created dynamically within the array, or they can be designated
for reuse for backup using a rotation strategy. Snapshots can also be designated for
use with the IR capabilities of Data Protector.
13 -26
UC434S F.00
Data protection
De-Duplication
De-Duplication
Because virtual tape libraries are disk-based backup devices with a virtual file system
and the backup process it tends to have a great deal of repetitive data, virtual tape
libraries lend themselves particularly well to data de-duplication. In storage
technology, de-duplication essentially refers to the elimination of redundant data. In
the de-duplication process, duplicate data is deleted, leaving only one copy of the
data to be stored. However, indexing of all data is still retained should that data ever
be required. De-duplication is able to reduce the required storage capacity since
only the unique data is stored.
With a virtual tape library that has de-duplication, the net effect is that, over time, a
given amount of disk storage capacity can hold more data than is actually sent to it.
To work de-duplication needs a random access capability offered by disk based
backup. This is not to say physical tape is dead, indeed tape is still required for
archiving and disaster recovery, both disk and tape have their own unique attributes
in a comprehensive data protection solution. The capacity optimization offered by
de-duplication is dependent on:
Backup policy, full, incremental
Retention periods
Data rate change
UC434S F.00
13 -27
The benefits of data de-duplication.
13 -28
The ability to store dramatically more data online (by online we mean disk
based)
An increase in the range of Recovery Point Objectives (RPOs) available data can
be recovered from further back in time from the backup to better meet Service
Level Agreements (SLAs). Disk recovery of a single files is always faster than
tape
A reduction of investment in physical tape by restricting its use more to a deep
archiving and Disaster recovery usage model
De-duplication can automate the disaster recovery process by providing the
ability to perform site to site replication at a lower cost. Because de-duplication
knows what data has changed at a block or byte level, replication becomes
more intelligent and transfers only the changed data as opposed to the complete
data set. This saves time and replication bandwidth and is one of the most
attractive propositions that de-duplication offers. Customers who do not use disk
based replication across sites today will embrace low-bandwidth replication, as
it enables better disaster tolerance without the need and operational costs
associated with transporting data off-site on physical tape. Replication is
performed at a tape cartridge level
UC434S F.00
Data protection
How Accelerated De-duplication Works

When the backup runs the data stream is processed as it is stored to disk assembling
a content database on the fly by interrogating the meta data attached by the backup
application. This process has minimal performance impact.
UC434S F.00
a.
After the first backup job completes, tasks are scheduled to begin the deduplication processing. The content database is used to identify subsequent
backups from the same data sources. This is essential, since the way objectlevel differencing works is to compare the current backup from a host to the
previous backup from that same host
b.
A data comparison is performed between the current backup and the

previous backup from the same host. There are different levels of
comparison. For example, some backup sessions are compared at an entire
session level. Here, data is compared byte-for-byte between the two versions
and common streams of data are identified. Other backup sessions
compare versions of files within the backup sessions. Note that within
Accelerated de-duplications object-level differencing, the comparison is
done AFTER the backup meta data and file system meta data has been
stripped away. (See the example in the following Figure 10) This makes the
de-duplication process much more efficient but relies on an intimate
knowledge of both the backup application meta data types and the data
type meta data (file system file, database file, and so on).
13 -29
13 -30
c.
When duplicate data is found in the comparison process, the duplicate

data streams in the oldest backup are replaced by a set of pointers to a
more recent copy of the same data. This ensures that the latest backup is
always fully contiguous, and a restore from the latest backup will always
take place at maximum speed.
d.
Secondary Integrity Checkbefore a backup tape is replaced by a deduplicated version with pointers to a more recent occurrence of that data, a
byte-for-byte comparison can take place comparing the original backup
with the reconstructed backup, including pointers to ensure that the two
are identical. Only when the compare succeeds will the original backup
tape be replaced by a version including pointers.
e.
Space reclamation occurs when all the free space created by replacing
duplicate data with pointers to a single instance of the data is complete.
This can take some time and results in used capacity being returned to a
free pool on the device
UC434S F.00
Data protection
The major issue with object-level differencing is that the device has to be
knowledgeable in terms of backup formats and data types to understand the Meta
data. HP Accelerated de-duplication will support a subset of backup applications
and data types at launch.
Additionally, object-level differencing compares only backups from the same host
against each other, so there is no de-duplication across hosts, but the amount of
common data across different hosts can be quite low.
The object-level differencing in HP Accelerated de-duplication is unique in the
marketplace. Unlike hash-based techniques that are an all-or-nothing method of deduplication, object-level differencing applies intelligence to the process, giving users
the ability to decide what data types are de-duplicated and allowing flexibility to
reduce the de-duplication load if it is not yielding the expected or desired results. HP
Object-level differencing technology is also the only de-duplication technology that
can scale to hundreds of terabytes with no impact on backup performance, because
the architecture does not depend on managing ever increasing amounts of Index
tables, as is the case with Hash based chunking. It is also well suited for larger
saleable systems since it is able to distribute the de-duplication workload across all
the available processing resources and can even have dedicated nodes purely for
de-duplication activities.
UC434S F.00
13 -31
How hash based chunking works

How hash based chunking works
How Dynamic De-duplication Works
13 -32
a.
As the backup data stream enters the target device (in this case the HP
D2D2500 or D2D4000 Backup System), it is chunked into nominal 4K
chunks against which the SHA-1 hashing algorithm is run. These results are
place in an index (hash value) and stored in RAM in the target D2D
device. The hash value is also stored as an entry in a recipe file which
represents the backup stream, and points to the data in the de-duplication
store where the original 4K chunk is stored. This happens in real time as the
backup is taking place. Step 1 continues for the whole backup data stream.
b.
When another 4K chunk generates the same hash index as a previous

chunk, no index is added to the index list and the data is not written to the
de-duplication store. An entry with the hash value is simply added to the
recipe file for that backup stream pointing to the previously stored data, so
space is saved. Now as you scale this up over many backups there are
many instances of the same hash value being generated, but the actual
data is only stored once, so the space savings increase.
UC434S F.00
Data protection
UC434S F.00
c.
In the case of backup 2. As the data stream is run through the hashing
algorithm again, much of the data will generate the same hash index codes
as in backup 1, hence, there is no need to add indexes to the table or use
storage in the de-duplication store. In this backup, some of the data has
changed. In some cases (#222, #75, and #86), the data is unique and
generates new indexes for the index store and new data entries into the deduplication store.
d.
And so the hashing process continues ad infinitum until as backups are

overwritten by the tape rotation strategy certain hash indexes are no longer
required, and so in a housekeeping operation they are removed.
13 -33
How hash-based chunking performs restores

How hash-based chunking performs restores
On receiving a restore command from the backup system, the D2D device selects the
correct recipe file and starts sequentially re-assembling the file to restore.
a.
Read recipe file.
b.
Look up hash in index to get disk pointer.
c.
Get original chunk from disk.
d.
Return data to restore stream.
e.
Repeat for every hash entry in the recipe file.
Issues Associated with Hash-Based Chunking - The main issue with hash-based
chunking technology is the growth of indexes and the limited amount of RAM storage
required to store them. Let us take a simple example: if we have a 1TB backup data
stream using 4K chunks, and every 4K chunk produces a unique hash value. This
equates to 250 million 20-byte hash values or 5GB of storage.
HP has developed a unique innovated technology leveraging work from HP Labs that
dramatically reduces the amount of memory required for managing the index without
sacrificing performance or de-duplication efficiency.
13 -34
UC434S F.00
Data protection
Despite the above limitations, de-duplication using hash-based chunking is a wellproven technology and serves remote offices and medium sized businesses very well.
The biggest benefit of hash-based chunking is that it is totally data formatindependent and it does not have to be engineered to work with specific backup
applications and data types. The products using the hash based de-duplication
technology still have to be tested with the various backup applications but the design
approach is generic.
UC434S F.00
13 -35
Pros & Cons of HP Dynamic Deduplication
What makes HP Dynamic De-duplication technology unique are algorithms

developed with HP Labs that dramatically reduce the amount of memory required for
managing the index, and without sacrificing performance or de-duplication
effectiveness. Specifically, this technology:
13 -36
Uses far less memory by implementing algorithms that determine which are the
most optimal indexes to hold in RAM for a given backup data stream
Allows the use of much smaller chunk sizes to provide more effective data
deduplication which is more robust to variations in backup stream formats or
data types
Provides intelligent storage of chunks and recipe files to limit disk I/O and
paging
Works well in a broad range of environments since it is independent of backup
software format and data types
UC434S F.00
Data protection
Enterprise Deployment with replication across remote and branch

offices back to data centers
Initially it will not possible for D2D devices to replicate into the much larger VLS
devices, since their de-duplication technologies are so different, but HP plans to be
able to offer this feature in the near future.
What will be possible is to replicate multiple HP D2D250 into a central D2D4000 or
replicate smaller VLS6200 models into a central VLS 12000 (See Figure 18)
De-duplication technology is leading us is to the point where many remote sites can
replicate data back to a central data center at a reasonable cost, removing the need
for tedious off-site vaulting of tapes and fully automating the processsaving even
more costs.
This ensures
The most cost effective solution is deployed at each specific site

The costs and issues associated with off site vaulting of physical tape are
removed
The whole Disaster recovery process is automated
The solution is scalable at all sites
UC434S F.00
13 -37
Hp Storage Works Disk to Disk and Virtual Library portfolio

with de-duplication
HP has a range of disk-based backup products with de-duplication starting with the
entry-level D2D2500 at 2.25TB user unit for small businesses and remote offices,
right up to the VLS12000 EVA Gateway with capacities over 1 PB for the high-end
enterprise data center customer. They emulate a range of HP Physical tape
autoloaders and libraries.
The HP StorageWorks D2D2500 and D2D4000 Backup Systems support HP
dynamic de-duplication These range in size from 2.25TB to 7.5TB and are aimed at
remote offices or small enterprise customers. The D2D2500 has an iSCSI interface to
reduce the cost of implementation at remote offices, while the D2D4000 offers a
choice of iSCSI or 4Gb FC.
The HP StorageWorks Virtual Library Systems are all 4Gb SAN-attach devices which
range in native user capacity from 4.4TB to over a petabyte with the VLS9000 and
VLS12000 EVA Gateway. Hardware compression is available on the VLS6000,
9000 and 12000 models, achieving even higher capacities. The VLS9000 and
VLS12000 use a multi-node architecture that allows the performance to scale in a
linear fashion. With eight nodes, these devices can sustain a throughput of up to
4800MB/sec at 2:1 data compression, providing the SAN hosts can supply data at
this rate. HP Virtual Library Systems will deploy the HP Accelerated de-duplication
technology
13 -38
UC434S F.00
Data protection
Remote replication
Remote replication
Synchronous replication
+ Guaranteed in-sync local and remote data
- Round-trip latency
Asynchronous replication
+ Improved application performance
- Remote data may not be fully updated
Synchronous replication
+ Guaranteed in-sync local and remote data
- Round-trip latency
Asynchronous replication
+ Improved application performance
- Remote data may not be fully updated
This is an example of Array based replication.
UC434S F.00
13 -39
HP StorageWorks Continuous Access EVA (CA EVA)

HP StorageWorks Continuous Access
EVA (CA EVA)
The maximum number of copy sets, DR groups, and remote copy sets is based on
the EVA storage system model and controller software version (for up to date details
check the HP StorageWorks Enterprise Virtual Array compatibility reference guide).
On all storage systems, the limit is the total number of DR groups and copy sets that
are either a source or a destination. When replicating across storage systems with
different limits, the lower limit applies to the storage system replication pair.
Hp StorageWorks Continuous Access (CA) EVA is a feature that allows data

replication between two or more EVAs. Data replication can be performed
synchronously or asynchronously, supporting interconnection technologies such as
FCIP and Fibre Channel, as well as also supporting bidirectional replication. Data
replication between sites is most widely used when creating a true disaster-tolerant
data center. A copy set is a replicated vdisk and a Data Replication (DR) group is a
data replication group that is comprised of replicated vdisks (Copy Sets). Each DR
group acts as a consistency group and all copy sets within that group share a single
write history log. All management tasks are performed on a DR group and not on
each copy set, this is because the DR group is the primary level of CA management
as such, all CA management actions are performed on the DR Group these include
tasks such as Write Mode, Failsafe Mode, Suspend Mode, and Failover. Replication
Solution Manager is the preferred software tool to manage BC and CA on EVA.
The number of replication group created should be tailored to the specific user
environment. The EVA provides a limited number of DR groups so understanding the
environment and its replication granularity requirements will help to reduce the
number of DR groups required and provide improved efficiency.
13 -40
UC434S F.00
Data protection
Synchronous replication (1 of 2)
Writes are ensured
Host A
1. I/O sent
to controllers
11
4. Acknowledgment
to host
2. Copy sent to
destination
Source array
5. I/O complete
3. Acknowledgment
from destination
Destination array
I/O completion status not returned to host until both local and remote writes to
cache complete
Data is mirrored in real-time
In-order delivery guaranteed using group sequence numbers
Data consistency is crucial to application
Can increase response time on writes
Higher latency of link, the more impact on performance
UC434S F.00
13 -41
I/O completion status not returned to host until both
local and remote writes to cache complete
Data is mirrored in real-time
In-order delivery guaranteed using group sequence
numbers
Data consistency is crucial to application
Can increase response time on writes
Higher latency of link, the more impact on performance
Note that the I/O complete message does not imply the data is on the disk platters.
Rather, it is in the battery protected writeback cache and is written to the disks when
not competing with I/O or before the cache is full.
13 -42
UC434S F.00
Data protection
Asynchronous replication (1 of 2)
Host A
1. I/O sent to
controllers
2. I/O
acknowledgement
3. I/O complete
5. Acknowledgment
from destination
4. Copy sent to
destination
1
Source array
Destination array
I/O completion status returned when local write completes
Destination writes are deferred until later
At risk if source system is lost
In-order delivery guaranteed using group sequence numbers
May improve performance
UC434S F.00
Response time on writes may decrease
Higher latency of link, the more performance may improve
13 -43
I/O completion status returned when local write
completes
Destination writes are deferred until later
At risk if source system is lost
In-order delivery guaranteed using group sequence

numbers
May improve performance
Response time on writes may decrease
Higher latency of link, the more performance may improve
13 -44
UC434S F.00
Data protection
Comparing replication modes

Comparing replication modes
Transaction is not complete
until this point
Synchronous
Perform
task
(user)
Update data
at source
Transmit
update
(network)
Update
Notify user
data at
(network)
destination
Time
Asynchronous
Perform
task
(user)
Target side updates can be

completed at a later time
Transaction is not
complete at this point
Update data
at source
Notify user
(network)
Transmit
update
(network)
Update
data at
destination
Time
HP recommends synchronous mode for most data replication. The reasons to use
synchronous or asynchronous modes are discussed in the best practices module.
Possible uses for each write mode

Synchronous
Financial transactions
Brokerage accounts
Banking
Mortgage servicing
Critical production data Websites updated
Telephone company databases
Bulk data movement
Manufacturing systems
Asynchronous
Data warehousing
Development systems
Mirrored websites
UC434S F.00
13 -45
13 -46
Index updates
Initial data center consolidation
UC434S F.00
Data protection
HP StorageWorks Storage Mirroring

HP StorageWorks Storage Mirroring
SAN to SAN
Remote Data Protection
Consolidation of storage
LAN
IP
IP
LAN
SAN Switch
IP
IP
Data is replicated
between servers
over LAN, WAN or
SAN IP connections.
SAN Switch
If your Disk Array does not support Controller-based Replication then software
replication can help.
Storage Mirroring can replicate across WAN links to offsite storage (no distance
constraints)
Because Storage Mirroring is based on standard IP technology, as IP over fibre
channel comes into maturity, the SAN can be used to take data replication off of the
LAN and onto the high speed SAN fabric. Also, with fibre channel currently limited
to 10 KM itself, Storage Mirroring can be used to replicate SAN based data over
WAN links to an offsite server or SAN.
UC434S F.00
13 -47
SWSM mirroring full

SWSM mirroring full
Source
Target
Initial Mirror
FULL Mirror provides target with baseline copy of data

Must complete before data consistency insured
ALL files (in repset) are sent to the Target. (32KB chunks)
Default in Connection Manager when building a connection.
Low CPU impact - High Memory impact on server. (compared
to File Diff)
High impact on Network resources. (compared to File Diff)
13 -48
FULL Mirror provides target with baseline copy of data
Must complete before data consistency insured
ALL files (in repset) are sent to the Target. (32KB chunks)
Default in Connection Manager when building a connection.
Low CPU impact - High Memory impact on server. (compared to File Diff)
High impact on Network resources. (compared to File Diff)
Process of transmitting user-specified data from the Source to the Target so that
an identical copy exists on the Target.
Initially connection, a Full Mirror of the selected data including file attributes and
permissions is mirrored to the Target
This creates a foundation upon which Storage Mirroring can effectively update
by replicating changes
By default mirroring occurs in 32KB chunks (MirrorChunckSize=32)
Low CPU utilization (compared to Diff mirror)
High Memory utilization
High Network utilization
UC434S F.00
Data protection
SWSM mirroring file difference

SWSM mirroring file difference
Source
Target
File Difference Mirror
File Difference Mirror syncs up data between Source and

Target
Compares DTS attributes of file on Source and its copy on Target.

Only files with DTS difference are sent to the Target. (32KB chunks)
Anytime a Source/Target connection is disconnected you must remirror!
High CPU impact - Low Memory impact on server. (compared to Full Mirror)
Low impact on Network resources. (compared to Full Mirror)
File Difference Mirror syncs up data between Source and Target
UC434S F.00
Compares DTS attributes of file on Source and its copy on Target.
Only files with DTS difference are sent to the Target. (32KB chunks)
Anytime a Source/Target connection is disconnected you must remirror!
High CPU impact - Low Memory impact on server. (compared to Full Mirror)
Low impact on Network resources. (compared to Full Mirror)
13 -49
Lab activity
Lab
activity
1. Module 13, Lab 1
Snapshot Management
2. Module 13, Lab 2
Installing HP
StorageWorks Storage
Mirroring
3. Module 13, Lab 3
SWSM Basic Operations
13 41uc434s c.01
41
13 -50
uc434s
uc434sDevelopment
d.00
d00 2009
2009
Company,
Hewlett-Packard
Hewlett-Packard
L.P.
Development
DevelopmentCompany,
Company,L.P.
L.P.
UC434S F.00
Performance
Module 14
Objectives
Objectives
Understand what factors affect storage area network
(SAN) performance
Describe Fibre Channel technology and how it affects
storage performance
List the factors that affect disk performance, such as data
rates and response time
Explain the effects of drive speed on performance
Plan a disk system that account for effects of RAID,
cache, and chunk size on performance
I/O profiling
UC434S F.00
14 -1
SAN performance objectives

SAN performance objectives
A SAN can be designed for
High performance
I/O per second
MB/s
High availability
Low cost
You cannot optimize all three

Understand what is important for the applications
This objective of a SAN is to access data. A SAN is often designed by drawing the
various components on a piece of paper and linking them with lines to represent the
Fibre Channel cables. Some questions that may arise are often answered without
considering performance. For example:
Question: How many disks are required?

Uninformed answer: Find out how much data the customer wants to store and
capacity of the largest disk drives. The calculation is then easy.
Question: What RAID level should be used?

Uninformed answer: How important is the customer data? If it is critical, then they will
need RAID 1 (Mirroring).
The preceding answers are too simplistic for designing a SAN to meet the
performance requirements of the customers application.
A SAN can be designed for:
1.
14 -2
High performance
a.
I/O per second
b.
MB/s
2.
High availability
3.
Low cost
UC434S F.00
Performance
But you cannot have all three, so you have to decide what is important for the
customer and the application.
When designing a SAN for performance, it means firstly having to determine the
performance requirements, and the application I/O profile. Even if the SAN is
already in production, you need to understand the application profile to determine
how you can optimize or maintain performance. Without this knowledge, any
change or upgrade could result in a lowering of performance.
Availability is also another consideration. But be aware that designing or
redesigning a SAN for availability will probably change the performance
characteristics of the SAN.
UC434S F.00
14 -3
Performance factors
Performance factors
Attenuation
Bandwidth
Data
rate
Applied load
Request size
Read/write ratio
Request
rate
Response Time
Rotational Speed
Seek Time
Service Time
Throughput
Utilization
Bus Contention
Complicating the task of delivering good performance is the lack of meaningful

performance measurement standards. Although vendors and analysts often cite
benchmarks such as high rates of I/O and cache hits or low seek times on disk
drives as examples of good performance, these statistics offer little insight to SAN
design.
A number of factors contribute to the difficulties an administrator has in gathering
and interpreting accurate performance data. The biggest issue is the collection of
performance statistics. This data may be spread across every component in the SAN
the database, the operating system, the host bus adapter (HBA), the switch, and
the storage array. SAN administrators need to verify that they have gathered all of
the data required before beginning analysis.
It is imperative to make sure the clocks of the components being measured are all
synchronized. A few minutes difference, or even a few seconds, on any of the
internal time clocks in the components could distort the interpretation of the results.
There are many tools available on the market that measure performance in a SAN.
However, performance-tuning software can create a new set of performance
problems by slowing down the SAN.
14 -4
UC434S F.00
Performance
Performance terms
The following definitions are frequently used to describe performance:
Attenuation - Loss of power specified in decibels per kilometer (dB/km).

Bandwidth total amount of data transferred through a medium or system per
unit of time.
Raw Bandwidth line speed Theoretical rate of a component, a transmission
line or bus has a physical signal rate that determines the maximum possible
data rate.
Sustained Bandwidth Is the data rate after accounting for all the overheads
that might be needed for the transmission operations. Sustained bandwidth will
always be less than the Raw Bandwidth if the transmission medium has
contention or if parts of the transmission are not considered to be useable data.
Sustained bandwidth is considered a better representation of the performance
factors of a component.
Data rate - The amount of data per unit of time moved across an I/O bus in the
course of executing an I/O load. The data rate varies according to the:
Applied load
Request size
Read/write ratio
The data transfer rate is usually expressed as megabytes per second (MB/s).
Considering bus arbitration and protocol overheads on the Ultra Wide SCSI bus, the
amount of data that can be processed is less than the rated bandwidth.
Example
The data rate for a Wide Ultra SCSI bus is approximately 38MB/s.
UC434S F.00
Request rate The number of requests processed per second. Workloads

attempt to request data at their natural rate. If this rate is not met, a queue builds
up at the processor and eventually saturation results.
Response time The time a device takes to process a request from issuance to
completion. It is the sum of wait time and service time. Response time is the
primary indication of performance and is typically expressed in milliseconds
(ms).
14 -5
Rotational speed The performance of a hard disk, measured in rotations per

minute (rpms). Rotational speed is important to the overall speed of the system
a slow hard disk can slow a fast processor. The effective speed of a hard disk
is determined by several factors.
Disk rpm is a critical component of hard drive performance because it directly

impacts the latency and the disk transfer rate. The faster the disk spins, the more data
passes under the magnetic heads that read the data; the slower the rpm, the higher
the mechanical latencies.
Seek time The time delay associated with reading or writing data to a disk
drive. To read or write data to a particular place on the disk, the read/write
head of the disk that must move to the correct place. This process is known as
seeking and the time it takes for the head to move to the right place is the seek
time.
Service time The amount of time a device needs to process a request. Service
time is also known as latency and varies with request characteristics.
Throughput The number of I/O requests satisfied per unit of time. Throughput
is expressed in I/O requests per second, where a request is an application
request to a storage subsystem to perform a read or write operation. Although
throughput and bandwidth are sometimes used interchangeably there is a
fundamental difference between the two terms. The difference stems from the
fact that operations reported in the throughput can have different data sizes.
Utilization The fraction (or percentage) of time a device is busy. Utilization
depends on the service time and request rate and is expressed as a percentage,
where 100% utilization is the maximum utilization.
Drive speed
Using faster drives yields better performance. How much of an increase depends on
the workload (I/O profile) and the applied load.
The original 1.6-inch 7,200-rpm drives consisted of up to ten 3.5-inch platters. When
spinning these drives at high rpms, interaction between the air and the platters
caused friction. This friction generated heat that caused drive failures. The 10,000rpm drive design was changed to use 2.8-inch platters that addressed the heat issue
and provided higher rotational speeds.
Bit density on the platters has also increased, reducing the total number of platters in
a drive and increasing drive and speed capacities.
The end result is:
14 -6
Higher capacity
Higher speed
Higher performing drives
UC434S F.00
Performance
Response time
Response time
Bottlenecks can form when data moves from a device with
a high data rate to a device with a lower data rate
Controller
bottleneck
Adapter
bottleneck
Load
imbalance
500MB/s
80MB/s
Host
CPU
Adapter
90MB/s
I/O Bus
55MB/s
Controller
Queues of request
Response time is a method of measuring the performance of a disk system. Most

vendors publish controller performance benchmarks that do not include response
times, and most technical literature is based on a response time of 50ms whereas an
optimum response time is typically 15ms or less.
The preceding graphic represents a computer system with a processor subsystem
rated at 500MB/s. Installed in the host is an HBA, which has a data rate of
80MB/s. The HBA is connected to a Fibre Channel Arbitrated Loop (FC-AL) that has
a data rate of 90MB/s. Attached to the FC-AL is disk controller that has a data rate
of 55MB/s.
If the host runs an application that can provide more than one outstanding I/O
request at a time, the response time is proportional to the bottlenecks that develop in
the system.
Bottlenecks (queues) can form when data moves from a device with a high data rate
to a device with a lower data rate. A load imbalance on the disk drives also affects
the service time of the disk device. RAID can be used to address the load imbalance
across drives, but other devices must be reconfigured to address bottlenecks.
In a single-threaded application environment, the response time depends on the sum
of the individual device service times.
UC434S F.00
14 -7
Bus utilization
Bus utilization
Bandwidth specifies the maximum rate at which data
can be transferred over that bus
Maximum data rate over a given bus is 85% of the bus
bandwidth
Apply the 80% rule to avoid excessive response times
Bus utilization should not exceed 68% of the maximum
bandwidth
Bus utilization is a factor to consider before beginning bottleneck analysis. The

bandwidth specifies the maximum rate at which data can be transferred over that
bus.
In practice, user data and other information are transferred over the bus. Additional
information that is transferred includes command and acknowledgement packets,
header information, and error detection and correction information.
The user data must then fit within the remaining bus bandwidth while maintaining an
effective data rate that is less than the specified bus bandwidth.
In most cases, the maximum user data rate over a given bus is 60 to 85% of the bus
bandwidth. Applying the 80% rule to avoid excessive response times, the bus usage
should not exceed 68% of the maximum bandwidth.
14 -8
Bus
Bandwidth
Maximum Data Rate
Fast SCSI-2
10MB/s
80% x 85% x 10 = 6.8MB/s
Fast-Wide SCSI-2
20MB/s
80% x 85% x 20 = 13.6MB/s
Wide-Ultra SCSI-2
40MB/s
80% x 85% x 40 = 27.2MB/s
Wide Ultra2 SCSI
80MB/s
80% x 85% x 80 = 54.4MB/s
Wide Ultra3 SCSI
160MB/s
80% x 85% x 160 = 108.8MB/s
UC434S F.00
Performance
Device utilization
Device utilization
Relationship between utilization and response time

10
Relative response time
9
8
7
6
5
4
3
2
1
0
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Utilization
The devices with the highest utilization can be identified after the performance data
has been collected. By definition, the devices with the highest utilization are the
bottleneck devices. The preceding graph depicts the relationship between utilization
and relative (normalized) response time.
When the incoming request rate is low (low utilization), the response time is equal to
the service time. This response time is given a value of 1 on the vertical axis. As
the workload increases (utilization increases), queues form within the I/O subsystem
and response time increases. At low utilization levels, the increase in response time is
relatively slow, but after the utilization rate exceeds 75%, the response time rises
rapidly.
Optimum response times are achieved when device utilization is kept below 80%.
This applies to all devices.
Improving performance
When the bottleneck devices have been identified, the next step to improve
performance is to take measures to reduce response times. Two ways to reduce
response time are:
Reduce the service time of the devices
Reduce usage of the devices
UC434S F.00
14 -9
Reducing service time

Reducing the service time often requires replacing the existing drives with newer
technology. The drive capacity section showed that 15,000 rpm provides the greatest
performance gain. It also showed that FC-AL drives do not provide major
performance gains over SCSI drives.
Another way to improve performance is to increase the number of queuing centers
(increase the number of spindles).
Reducing usage
Reduce the device request rate or the device service time to lower usage.
You can reduce the device request rate by:
14 -10
Distributing the workload over more drives by using a drive array and RAID
technology
Shifting the workload to another device if the application permits
Bypassing the device with cache
UC434S F.00
Performance
SAN performance Considerations

SAN performance considerations
The following SAN components affect SAN application

performance:
Host CPUs
Fibre Channel HBAs
SAN topology and the number of fabrics

I/O transfer sizes and usage patterns
RAID controllers
Disk configuration
A single-switch fabric provides the highest level of performance. In a fabric with

multiple switches, the following factors can affect performance:
Latency
Switch latency is less than 5% (at 1 Gb/s) of the data transfer time; therefore, the
number of switches and hops between devices is not a major performance factor.
However, as devices send frames through more switches and hops, other data traffic
in the fabric routed through the same ISL or path can cause oversubscription.
Oversubscription
Oversubscription degrades Fibre Channel performance. When devices must contend
for the same
ISL or path, each device receives an equal share or 1/nth of the available bandwidth
on the path (where n is the number of contending devices). Oversubscription occurs
when one or more devices sends more data than the total bandwidth available on
the ISL or path.
Fabric interconnect speeds
Fibre Channel supports 8 Gb/s, 4 Gb/s, 2 Gb/s, and 1 Gb/s speeds. For optimum
performance, configure a fabric with all components at the same, highest available
speed. Additional factors such as distance, number of switch and device port buffers,
and device response times can also affect performance.
Mixed Fibre Channel speeds

UC434S F.00
14 -11
For fabrics consisting of 8 Gb/s, 4 Gb/s, and 2 Gb/s or 4 Gb/s, 2 Gb/s, and 1
Gb/s switches and devices, the fabric segment connections negotiate the speed at
which specific devices communicate.
The presence of lower speed devices in a fabric does not force other independent
higher speed devices or paths to a lower speed. Fibre Channel requires that all 8
Gb/s ports be able to negotiate to 4 Gb/s and 2 Gb/s, and all 4 Gb/s ports to 2
Gb/s and 1 Gb/s speeds. Switch ports or user ports in a fabric communicate at the
highest mutually supported speed.
14 -12
UC434S F.00
Performance
Latencies
Latencies
Long distance fiber
Host
8?
16
Switch
16
2
Array
16
27
Switch
Switch
27
Buffer processing time also an issue

Credits adjustable on some switches
16
2
Array
Buffer-to-Buffer (BB) credit flow control is implemented to limit the amount of data that
a port may send based on the number and size of the frames sent from that port.
Buffer credits represent finite physical port memory. Within a fabric, each port may
have a different number of BB credits.
Within a connection, each side may have a different number of BB credits.
Buffer-to-Buffer flow control is flow control between adjacent ports in the I/O path,
for example, transmission control over individual network links. A separate,
independent pool of credits is used to manage Buffer-to-Buffer flow control. Buffer-toBuffer flow control works by a sending port using its available credit supply and
waiting to have the credits replenished by the port on the opposite end of the link.
These BB credits are used by Class 2 and Class 3 service and rely on the Fibre
Channel Receiver-Ready (R_RDY) control word to be sent by the receiving link port to
the sender. The rate of frame transmission is regulated by the receiving port based on
the availability of buffers to hold received frames.
Upon arrival at a receiver, a frame goes through several steps. It is received,
deserialized, decoded, and is stored in a receive buffer where it is processed by the
receiving port. If another frame arrives while the receiver is processing the first frame,
a second receive buffer is needed to hold this new frame. Unless the receiver is
capable of processing frames as fast as the transmitter is capable of sending them, it
is possible for all of the receive buffers to fill up with received frames. At this point, if
the transmitter should send another frame, the receiver will not have a receive buffer
available and the frame will be lost. Buffer-to-Buffer flow control provides consistent
and reliable frame delivery of information from sender to receiver.
UC434S F.00
14 -13
The default configured F_Port buffer credit is fixed at eight buffers. You can use the
portCfgFPortBuffers command to configure a given port with the specified number of
buffers.
14 -14
UC434S F.00
Performance
ISL oversubscription
ISL oversubscription
3 to 1 ISL oversubscription
12
x1
Gb
/s
Po
rts
4 x 1 Gb/s ISLs
In many cases, ISL over-subscription is not a performance-limiting factor in SAN

design. Storage port fan-out, low application I/O requirements, and performance
limits on edge devices are more likely to be the focus for performance improvement.
You can usually use the same ISL oversubscription ratio used for storage port fan-out.
(This is usually around 7:1.) Understanding ISL oversubscription at a detailed level
enables you to better analyze performance issues using performance models. When
all ports operate at the same speed, ISL oversubscription is the ratio of node or data
input ports that might drive I/O between switches to the number of ISLs over which
the traffic could cross.
In the preceding illustration, the oversubscription ratio on the left-most switch is three
node ports to one ISL. This is usually abbreviated as 3:1. Twelve hosts are
connected to the upper-left edge switch and only four ISLs are connected to the core.
Thus, each ISL has three hosts.
If all of these hosts tried to simultaneously use the ISLs at full speedeven if the hosts
were accessing different storage deviceseach would receive only about one-third
of the potential bandwidth available. The simple oversubscription formula is:
ISL Over-Subscription = Number of Nodes/Number of ISLs, or Io=Nn/i.
This is reduced as a fraction so that Ni:1.
UC434S F.00
14 -15
Bandwidth consumption and congestion

An oversubscribed link is one on which multiple devices might contend for
bandwidth. A congested link is one on which multiple devices actually are
contending for bandwidth. For years, traditional data networks have been built with
very high levels of oversubscription on links. The Internet is probably the best-known
example of this. It has links that are oversubscribed at a rate of millions to one.
Although most SANs cannot support Internet-like oversubscription ratios, they can
have several characteristics that enable them to function well even with oversubscribed links. These characteristics include burst traffic, shared resources, and
low peak usage by devices, good locality, and devices that can generate only a
small fraction of the I/O as compared to the available bandwidth. Most networks
have all of these characteristics to some degree. Moreover, organizations can often
realize substantial cost savings by deliberately designing a SAN with a certain
amount of oversubscription.
When performance service levels are critical and the bandwidth requirements are
high, lower oversubscription levels or traffic localization should be targeted.
14 -16
UC434S F.00
Performance
Hop latency
Hop latency
The
time it takes a frame to traverse from its source

to its destination, is referred to as the latency of the
link
Negligible impact on I/O
For most I/O profiles, hop-count latency is
inconsequential from both a switch latency and optical
latency standpoint because the millisecond disk I/O
latency is greater than the microsecond latency of a
Fibre Channel fabric
Fabric latency the time it takes a frame to traverse from its source to its destination,
is referred to as the latency of the link. Sometimes a frame is switched from source to
destination on a single switch, and other times a frame must traverse one or more
hops between switches before it reaches its destination. A common misconception is
that the hop counts introduce unacceptable latency. For most Fibre Channel devices,
the latency associated with traversing one or more ISLs is inconsequential.
Example:
Every hop in the B-Series SAN fabric adds no more than 2ms of latency. In a large
fabric designed with seven hops between two devices (the B-Series-supported
maximum), the latency could be up to 14ms.
The distance between switches also introduces latency, especially for long-distance
solutions spread over larger metropolitan areas. The speed of light in fibre optics is
approximately five u/s per km.
B-Series switches address the need for long-distance performance with B-Series
Extended Fabrics. This product enables full-bandwidth performance across long
distances spanning more than 100km, with greater distances possible at lower
speeds.
UC434S F.00
14 -17
Negligible impact on I/O

For most I/O profiles, hop-count latency is inconsequential from both a switch latency
and optical latency standpoint. This is because the millisecond disk I/O latency is
greater than the microsecond latency of a Fibre Channel fabric. Because it is so
small, virtually no applications are affected by the added latency.
Hop latency is not a reason to keep hop counts low in a SAN design. However, to
reduce the risk of oversubscription in a SAN, eliminate all switch hops (localized
traffic). The second-best scenario is actually two hops, rather than the more intuitive
one hop because a two-hop design enables fabric shortest path first (FSPF) to
perform better load sharing across multiple ISLs.
14 -18
UC434S F.00
Performance
Data Priority Quality of Service

Data Priority Quality of Service
Enables traffic
prioritization during
network congestion
User-definable,
weighted queues
Priority is given to
control traffic over
all others
Map to IP QoS for
SAN Extension
DWRR
Weight
Priority
Absolute
Queue 2
60
Queue 3
30
Queue 4
10
PQ
DWRR 2
Transmit
Queue
DWRR 3
DWRR 4
Cisco achieves QoS by using a Deficit Weighted Round Robin (DWRR).

A scheduler which is configured in the first switch and ensures that high priority traffic
is treated better than low priority traffic. For example, DWRR weights of 60:30:10
implies that the high priority queue is serviced at 6 times the rate of the low priority
queue. This guarantees lower delays and higher bandwidths to high priority traffic if
congestion sets in. A similar configuration in the second switch ensures the same
traffic treatment in the other direction. If the ISL is congested when the OLTP server
sends a request, the request is queued in the high priority queue and is serviced
almost immediately as the high priority queue is not congested. The scheduler
assigns it priority over the backup traffic in the low priority queue.
Note that the absolute priority queue always gets serviced first; there is no weighted
round robin.
Brocade use Virtual Channels within each ISL. VC_RDY flow control differentiates
traffic across an ISL. It serves two main purposes: to differentiate fabric internal traffic
from end-to-end device traffic, and to differentiate different data flows of end-to-end
device traffic to avoid head-of-line blocking. Fabric internal traffic is generated by
switches that communicate with each other to exchange state information such as link
state information for routing and device information for Name Service. This type of
traffic is given a higher priority so that switches can distribute the most up to- date
information across the fabric even under heavy device traffic. Additionally, multiple
I/Os are multiplexed over a single ISL by assigning different VCs to different I/Os
and giving them the same priority (unless QoS is enabled).
UC434S F.00
14 -19
Each I/O can have a fair share of the bandwidth, so that a large-size I/O will not
consume the whole bandwidth and starve a small-size I/O, thus balancing the
performance of different devices communicating across the ISL.
14 -20
UC434S F.00
Performance
Device attachment points

Device attachment points
Although device placement does not constitute a fabric topology, it can affect and be
affected by the type of topology selected.
For example, by attaching a device to a core switch you reduce the quantity of core
ports available for expansion. Expansion issues are less of a concern for the higher
port count B-Series, M-Series, and C-Series SAN switches.
Local Attach Adding a host and its disk to one edge switch consumes no core
ports and no hops are involved in transferring data.
Core Attach Adding a host to an edge switch and its disk to a core switch
introduces one hop when transferring data and consumes one core port.
Edge Attach Adding a host and its disk to different core switches increases the
number of hops to two when transferring data and consumes two core ports.
UC434S F.00
14 -21
Place fastest switches in the core
2Gb/s End to End
Not 2Gb/s End to End
Place fastest switches in the core
2Gb/s ISL
4Gb/s connect
8Gb/s Trunk
Localize 2 Gb/s
devices
Maximum distance for a 8Gb/s ISL is 150 meters using OM3 Cable
A switch that supports auto-sensing of 1Gb/s, 2Gb/s,4Gb/s and 8Gb/s device

connections, such as the B-Series, M-Series, or C-Series switches, introduces many
benefits and choices to the SAN designer. As SAN devices evolve from 1Gb/s to
2Gb/s to 4Gb/s to 8Gb/s capable, designing such capability into a SAN will
extend the life span of the initial design.
As an interconnect between switches, 8Gb/s ISLs deliver high performance. Other
devices that are not 8Gb/s capable can still benefit from the 8Gb/s capabilities of a
switch because to combine multiple 1Gb/s connections over a 8Gb/s ISL or trunk.
The advent of 8Gb/s ISLs essentially increases the performance by a increase of
eight compared to a similarly designed SAN built with 1Gb/s ISLs and nodes. If
required, you can scale back the number of ISLs to yield additional ports for device
connections. Trunking amplifies this performance benefit, because the ISLs are now
faster and used more efficiently.
When designing a SAN with 8Gb/s switches, the same guidelines that apply to
trunking apply to 8Gb/s capabilities. Place these switches adjacent to each other to
take advantage of 8Gb/s ISLs.
For core/edge topologies, place trunking-capable switches in the core. If 8Gb/s
connectivity is required, you can attach these devices to the 8Gb/s core switches if
8Gb/s edge switches are not yet implemented. By placing 8Gb/s switches in the
core, it ensures that a 8Gb/s path exists end to end.
14 -22
UC434S F.00
Performance
If a significant number of 8Gb/s devices are required and the performance

requirements are high, an effective strategy is to localize the 8Gb/s devices on the
same switch or group of switches.
UC434S F.00
14 -23
Distance considerations
Distance considerations
2.0
0.5KB
Response time (ms)
1.8
1.6
1KB
1.4
4KB
1.2
1.0
8KB
0.8
0.6
0.4
0.2
0.0
0 km
10.5 km
21 km
31.5 km
42 km
Length of fiber connection
Latencies increase
With distance because of the speed of light
With transfer size, so more trips are needed
If there are no available buffer credits
Buffer credit delays result in an underutilized fiber
Contention occurs when two sessions that are sharing an ISL try to send full frames at the same
time
Each session receives only half the potential bandwidth
The response time of different transfer sizes over different lengths of fiber are shown
in the graph.
When designing a Fibre Channel SAN, consider:
Latencies increase with distance because of the speed of light.
Latencies increase with transfer size, so more trips are needed.
Latencies increase if there are no available buffer credits.
Buffer credit delays result in an underutilized fiber.
Contention occurs when two sessions that are sharing an ISL try to send full
frames at the same time. Each session receives only half the potential bandwidth.
Note: Buffer credits are adjustable on the HP supported fabric switches.
14 -24
UC434S F.00
Performance
Maintaining performance in an extended SAN

beyond 5 or 10km
Maintaining performance in an
extended SAN beyond 5 or 10km
Primary
consideration
Maintaining performance of ISL connection

Flow control mechanism = Buffer-to-buffer credits
The number of buffer-to-buffer credits should be
increased when distance between switches exceeds
10km for a 1Gb
5km for a 2Gb
1Gbps link = 56 buffer-to-buffer credits for 100km

distance
2Gbps link = One buffer-to-buffer credit for each 1km
distance
Primary consideration
Maintaining performance of ISL connection
Flow control mechanism = Buffer-to-buffer credits
The number of buffer-to-buffer credits should be increased when distance

between switches exceeds
10km for a 1Gb
5km for a 2Gb
1Gbps link = 56 buffer-to-buffer credits for 100km distance
2Gbps link = One buffer-to-buffer credit for each 1km distance + 6
4Gbps link = Two buffer-to-buffer credit for each 1km distance + 6
8Gbps link = Four buffer-to-buffer credit for each 1km distance + 6
UC434S F.00
14 -25
Distributed fabrics
Distributed fabrics
Extended Fabrics
1
8
Switch
32.8km fiber
Switch
16
15
14
13
12
11
10
Buffer credit starvation

1
6
Switch
41.0km fiber
Switch
16
15
14
13
12
11
10
or
Switch ports run out of buffer to buffer credits
as distance increases
Buffer-to-buffer credit management affects performance over distances; therefore,

allocating a sufficient number of buffer credits for long-distance traffic is essential to
performance. To prevent a target device (either host or storage) from being
overwhelmed with frames, the Fibre Channel architecture provides flow control
mechanisms based on a system of credits.
Each of these credits represents the ability of the device to accept additional frames.
If a recipient issues no credits to the sender, no frames can be sent. Pacing the
transport of subsequent frames on the basis of this credit system helps prevent the loss
of frames and reduces the frequency of entire Fibre Channel sequences needing to
be retransmitted across the link.
Because the number of buffer credits available for use within each port group is
limited, configuring buffer credits for extended links may affect the performance of
the other ports in the group used for core-to-edge connections. You must balance the
number of long-distance ISL connections and core-to-edge ISL connections within a
switch. Configuring long-distance ISLs between core and edge switches is possible,
but is not a recommended practice.
With the exception of 3xxx series and earlier switches, all switch ports provide
protection against buffer depletion through buffer limiting. A buffer-limited port
reserves a minimum of eight buffer credits, allowing the port to continue to operate
rather than being disabled due to a lack of buffers.
The eight buffer minimum allows 4 and 8 Gbps speeds over distances within most
data centers. Buffer-limited operations are supported for the LS and LD extended ISL
modes only. For LD, distance in kilometres is the smaller of the distance measured
14 -26
UC434S F.00
Performance
during port initialization versus the desired distance value. For LS, distance in
kilometres is always the desired distance value.
Long distance link modes

To support long distance links use the portCfgLongDistance command, this
command allocates a sufficient numbers of full size frame buffers on a particular

port. Changes made by this command are persistent across switch reboots and
power cycles. This command supports the following long-distance link modes:
Static Mode (LO) - L0 is the normal (default) mode for a port. It configures the
port as a regular port. A total of 20 full-size frame buffers are reserved for data
traffic, regardless of the ports operating speed; therefore, the maximum supported
link distance is up to 10 km at 1 Gbps, up to 5 km at 2 Gbps, up to 2 km at 4 Gbps,
and up to 1 km at 8 Gbps.
Static Mode (LE) - LE configures an E_Ports distance greater than 5 km and up to
10 km. LE does not require an Extended Fabrics license. The baseline for the
calculation is one credit per km at 2 Gbps. This yields the following values for 10
km:
- 5 credits per port at 1 Gbps.
Dynamic Mode (LD) - LD calculates BB credits based on the distance measured
during port initialization. Brocade switches use a proprietary algorithm to estimate
distance across an ISL. The estimated distance is used to determine the BB credits
required in LD (Dynamic) extended link mode based on a maximum Fibre Channel
payload size of 2,112. An upper limit can be placed on the calculation by
providing a desired distance value. Fabric OS confines user entries to no larger than
what it has estimated the distance to be. When the measured distance is more than
desired distance, the desired distance (the smaller value) is used in the calculation.
Static Long-Distance Mode (LS) - LS calculates a static number of BB credits
based only on a user-defined desired distance value. LS mode also assumes that all
FC payloads are 2112 bytes. Specify LS mode to configure a static long distance
link with a fixed buffer allocation greater than 10 km. Up to a total of 1452 full-size
frame buffers are reserved for data traffic, depending on the specified desired
distance value.
NOTE
With the introduction of FOS 6v6.x Long distance modes L0.5, L1, and L2 are not
supported.
Use the Enter the portBufferShow command to display the remaining buffers in a port
group.
UC434S F.00
14 -27
Extended distance topology

Extended distance topology
Site A
0 120 KM
Site B
This is the recommended topology to use when connecting two geographically

separate sites. The fabric maximum size is 392 ports when using 16-port switches.
You also can build a smaller fabric using this topology. Scaling performance by
adding ISLs requires a smaller configuration or the replacement of the existing
complex core with larger core switches. To maintain performance, locality within
each location is necessary, as the bandwidth between locations is minimal. Note
that ISL oversubscription within a location is recommended to be no more than a 3:1
ratio but up to 7.1 can be supported.
14 -28
UC434S F.00
Performance
Performance Guidelines within the SAN

Performance guidelines within the SAN
Implement dual-fabric SANs.
In a cascaded or core-edge fabric, position switches with the highest

port speeds near the center of the fabric.
Use the highest speed available for all infrastructure components and
devices.
Ensure that communicating devices have the same speed

connectivity path through the fabric.
Connect devices that communicate frequently to the same Fibre

Channel switch.
use the same fabric topology and configuration in both fabrics to

maintain balanced SAN performance.
When possible, ensure that there is an equal number of highbandwidth application servers and storage systems (for one-to-one
access).
Ensure that FCC is enabled on all C-series switches.
use separate HBAs for disk and tape connections.
Although the topology and size of the fabric affect performance, adhering to the
rules and recommendations outlined in this guide minimizes these factors. The
topology designs have been defined to accommodate specific data access types.
Recommendations on the number of ISLs based on device-to-device access ratios
ensure that adequate bandwidth is available across the fabric, minimizing
oversubscription.
To maximize fabric performance, HP recommends the following guidelines:
Implement dual-fabric SANs.
In a cascaded or core-edge fabric, position switches with the highest port speeds
near the center of the fabric.
Use the highest speed available for all infrastructure components and devices.
Ensure that communicating devices have the same speed connectivity path through
the fabric.
Connect devices that communicate frequently to the same Fibre Channel switch.
When possible, ensure that there is an equal number of high-bandwidth
application servers and storage systems (for one-to-one access).
Ensure that FCC is enabled on all C-series switches.
FCC allows C-series switches to intelligently regulate traffic across ISLs and ensure
that each initiator- target pair of devices has the required bandwidth for data
transfer. C-series switches can also prioritize frames using the QoS feature.
UC434S F.00
14 -29
Determining the required bandwidth

Determining the required bandwidth
Collect the peak read and write workloads for a given period of time
At each sample interval, capture reads per second (I/Os per second),
read throughput per second (Mb/s), writes per second (I/Os per second),
and write throughput per second (Mb/s).
If possible, collect read and write latency data.
Perform the collection by application, capturing the data for each logical
unit (device) used by that application.
Create a graph of each data set that shows where the peaks occur
during the day
Determine whether the daily average change rate is level or is in bursts
You can determine the required bandwidth for any application.

1. Collect the peak read and write workloads for a given period of time. For
Windows operating systems, use a tool such as PERFMON to capture the current
performance requirements without Continuous Access EVA operating. Similar tools
exist for other operating systems.
At each sample interval, capture reads per second (I/Os per second), read
throughput per second (Mb/s), writes per second (I/Os per second), and write
throughput per second (Mb/s).
If possible, collect read and write latency data.
Perform the collection by application, capturing the data for each logical unit
(device) used by that application.
2. Create a graph of each data set that shows where the peaks occur during the
day.
Determine whether the daily average change rate is level or bursty.
Consider how these numbers will increase over the next 12 to 18 months.
The results of this scaling process become your design goal.
14 -30
UC434S F.00
Performance
If dealing with Disaster recovery solutions consider the following:

Determine the values for RPO (Recovery Point Objective) and RTO (Recovery Time
Objective).
RPO measures how much data is lost due to a problem at the source site. By
definition, an RPO of zero (no data can be lost) requires synchronous replication,
regardless of which data replication product you use.
RTO indicates when to start using the recovery site. This measurement includes
data about application failover and restart.
For asynchronous Continuous Access EVA, the RPO design space is near zero.
XP Continuous Access Asynchronous supports an RPO from near zero to many
hours.
Continuous Access EVA and XP Continuous Access all synchronous replication with
an
RTO equal to zero.
3. Once the data has been collected:
If the RPO is near zero, use the peak write rate and throughput to estimate the
bandwidth you need. For some real-time applications (such as Microsoft Exchange),
increase the bandwidth between 2 to 10 times this initial estimate due to wait time
for link access.
If the RPO is greater than zero, then average the change rate over the RPO interval
and use this value as an estimate of the inter-site bandwidth. You might need to
increase or decrease this bandwidth, depending on the environment and the amount
of time needed to complete the last write of the day before starting the next day's
work.
UC434S F.00
14 -31
Drive selection and performance
Characteristics that affect drive performance include:
Bus type
Seek time
Drive rotation speed
Access pattern or workload
Random (requires high I/O per second capability)
Sequential (requires high MB/s capability)
Flow control
Drive scaling
Moving to faster interfaces does not always improve disk performance (response
time). The graph shows the amount of time required to transfer 8KB of data from
a drive.
A full Fibre Channel frame takes 20 microseconds to pass from the beginning of the
transmission to the end of the transmission. One 2KB I/O can fit into one frame.
Based on the propagation of the Fibre Channel media, the beginning of the frame is
4km away when the end leaves the HBA. That means that for most fabrics, only one
frame is in transit at any given time.
14 -32
UC434S F.00
Performance
The switch adds less than a 2 microsecond delay, and if one frame must wait for
another to finish traversing an inter switch link (ISL), the wait is 20 microseconds, or
about 20 times the actual time of traversing a switch.
The most significant time delays occur within the disk drive because it is a
mechanical device. Although the amount of data that can be stored on a disk
continues to increase and the physical size of the disk continues to decrease, the time
to perform a seek and the rotational speed of disk drives continue to limit disk drive
performance. The preceeding graph shows that changing interfaces results in
marginal performance gains, as indicated by the last time slice.
SCSI and Fibre Channel drives use the same state-of-the-art mechanicals. Both have
the same rotational and seek characteristics.
However, changing from SCSI drives to an FC-AL configuration would not provide
significant performance gains. On average, performance increases are minimal
compared to regular SCSI drives for 8KB I/O.
A major benefit of Fibre Channel technology is the distance allowed between the
controller and the drives. Extended distances can be implemented with SCSI drive
subsystems using Fibre Channel interfaces between the host and storage system.
Fibre Channel drives provide the greatest benefit in high bandwidth applications such
as video editing and data streaming.
Bottleneck analysis is the best way to improve response time. To perform this
analysis, determine which aspect of your system workload provides the most stress
and address that issue first.
UC434S F.00
14 -33
RAID and RAID selection
RAID technology consists of a series of independent disk drives controlled by a

common array management component. RAID can be implemented at software and
hardware levels. To overcome the performance and availability limits of disk drives.
There are several levels of RAID because no RAID level can optimize the following
three factors:
Performance
Price
Availability
When designing a SAN, you must determine which of these factors is most important
for the customer and use the corresponding RAID level.
The different RAID levels affect read and write performance in different ways. For
example, RAID 5 can significantly improve read performance but significantly reduce
write performance. Understanding the read/write ratio of an application will
determine if a particular RAID level will increase or a decrease performance.
14 -34
UC434S F.00
Performance
RAID levels
RAID level comparisons to a single disk and respective uses are:
RAID 0 Striping Provides I/O load balancing but does not provide protection.
RAID 0 is good for testing and benchmarking or when data can easily be recovered.
It is the fastest of all RAID levels for both reads and writes.
RAID 1 Mirroring Provides performance improvement in a read-intensive
environment. If the environment is write-intensive, then performance will be reduced.
Parallel reads can provide more than a 10% performance increase, depending on
data patterns.
RAID 0+1 Striped Mirror Is used when continuous performance and availability
are required. This level combines the benefits of RAID 1 and RAID 0, providing load
balancing and parallel reads. However, it is the most expensive RAID level to
implement.
RAID 5 Independent access, parity striped over all the drives Provides balancing
and parallel reads. However, RAID 5 requires a read/write ratio greater than 55% to
be effective. RAID 5 can be used in a high-performance environment if sufficient
bandwidth can be provided for the application.
RAID 6 According to the Storage Networking Industry Association (SNIA), the
definition of RAID 6 is: "Any form of RAID that can continue to execute read and
write requests to all of a RAID array's virtual disks in the presence of any two
concurrent disk failures. RAID 6 does not have a performance penalty for read
operations, but it does have a performance penalty on write operations because of
the overhead associated with parity calculations. Performance varies greatly
depending on how RAID 6 is implemented in the manufacturer's storage architecture
RAID selection and planning

The following table summarizes the characteristics of each RAID level and the best
application environments using HP controller-based storage systems.
UC434S F.00
14 -35
RAID level efficiency

RAID level efficiency
Log files
Relative
efficiency
Relative Efficiency
120
120
Database files
0
100
100
80
RAID 0
RAID 1+
0+10
RAID 5
10 ++10
60
60
40
40
20
20
00
0%
0%
20%
20%
40%
60%
40%
60%
%
read
operations
% Read Operations
80%
80%
100%
100%
The preceding graph compares RAID 0, 1+0, and 5 under different read-to-write
ratios. Log files consist entirely of write requests during normal operation and are
positioned at the 0% read mark. Database files, however, can vary in the level of
reads to writes depending on the given environment.
The graph shows that if an application consists of 100% random access reads, the
relative performance is similar for all specified RAID levels. RAID overhead is only
relevant when the application starts writing to disk. The performance of the system is
limited to the number of drives.
RAID 0 provides no protection. RAID 1+0 provides the best performance, with RAID
5 being the next best level.
14 -36
UC434S F.00
Performance
Disk Performance
Users are interested in the number of transactions they can perform within a set
amount of time.
Choosing enough disk drives of the correct type can provide a certain ratio of MB/s
or I/O per second but this does not provide the number of user transactions. The
RAID controller generates multiple read/writes to the disks based on one read/write
request from the application, and the application generates multiple read/write
requests to the RAID controller based on one read/write request from the User
transaction.
UC434S F.00
14 -37
Planning a disk system

Planning a disk system
1.
Use disk arrays
2.
Physically separate sequential and random data
3.
Distribute data files over as many disk drives as necessary
4.
Do not use RAID 5 in write-intensive environments
5.
Use hardware RAID 1 for the redo log files and hardware RAID
5 for the data files
6.
Balance I/O between arrays
7.
Spread your disks over as many RAID controller channels as

possible
Understanding the major characteristics of an environment and the associated

performance goals helps you determine the required disk system configuration.
Rule 1 Use disk arrays

Benefits of an array controller include:
Safe write caching through the array accelerator
Superior manageability such as multiple volumes with different RAID levels
High flexibility such as separation of I/O into different disk volumes
Hardware RAID, automatic recovery, hot-plug capability, and online spares
Automatic data distribution and I/O balancing across multiple disk drives
Superior performance and I/O optimization
Rule 2 Physically separate sequential and random data

Performance suffers the most when the redo log and data files exist on the same
physical volume. The redo log is sequential in nature, whereas most of the data file
access is random. To avoid excessive head movement of the disk drives, place the
redo log on a physically separate volume from the data files.
Also, having more than one redo log on a physical volume can change the write
profile to the disk from sequential to random as the applications write to one log and
14 -38
UC434S F.00
Performance
then to the other. Until the redo log has been updated, the transaction is not
complete. Try to use different physical volumes for each redo log.
Note: Multiple sequential I/O requests to the same physical volume randomize the
I/O and decrease performance. Place multiple sequential I/O streams on separate
physical volumes to achieve the best performance.
Rule 3 Distribute data files over as many disk drives as necessary

Performance suffers when there are insufficient drive spindles. Many customers buy a
small number of large-capacity drives as opposed to a larger number of low-capacity
drives because of cost and management concerns, and because they do not
understand performance implications.
The RAID section described how RAID level and the number of drives affect the
available I/O bandwidth for a given disk configuration. To avoid a disk-bound
server, have enough drives (regardless of the drive capacity) to sustain the I/O rates
of the application.
Rule 4 Do not use RAID 5 in write-intensive environments

RAID 5 has a greater write overhead than any other level of RAID, which slows the
overall performance of the system. For every write request, the drive array controller
can issue four I/O requests:
Request to read the old data block
Request to read the old parity block
Request to write the new data block
Request to write the new parity block
Always use RAID 1+0 in an environment that requires consistent performance.

RAID 5 has benefits such as a low RAID overhead from the capacity perspective.
RAID 5 can be a viable alternative if adequate disk bandwidth is provided for the
required level of performance. RAID 5 used in conjunction with write-back (batterybacked) cache usually provides adequate performance for most environments.
Rule 5 Use hardware RAID 1 for the redo log files and hardware RAID
5 for the data files
For performance, recovery, and other benefits, use hardware RAID whenever
possible. The guidelines are:
Protect the redo log files and place them on a RAID 1 volume.
Place the data files on a RAID 5 volume if no downtime from a failed disk is
required.
Configure RAID 5 volumes to cache writes.
UC434S F.00
14 -39
Rule 6 Balance I/O between arrays

Distribute the data evenly among many disk drives for best performance. This data
distribution can be achieved with an array controller that stripes data across the
attached drives. Although this method ensures even load distribution within an array,
an imbalance between arrays is likely to occur.
Some applications permit data files to be spread across several drives or volumes to
balance I/O between arrays. In-depth knowledge of the application behavior is
essential for effective load balancing.
Rule 7 Spread your disks over as many RAID controller channels as

possible
Spread the disk drives over as many SCSI buses or Fibre Channel loops as possible;
otherwise, writes to the data drive and RAID parity drive will be done in series rather
than in parallel.
14 -40
UC434S F.00
Performance
Data caching technologies

Data caching technologies
Application caches
System
System
File system caches

Controller caches
Device caches
Controller
Caching affects performance

It reduces disk access (with cache hits)
It reduces the negative effects of the RAID overhead
It assists in disk I/O request sorting and queuing
Caching exists at different levels within a system to increase performance. Caching

technologies usually consist of high-speed memory that minimizes the queues in a
system. In multiprocessor systems, caching is used within the processors to reduce
access to main memory, which helps reduce memory contention. The closer the
cached data is to the application, the more performance improves.
Example:
If it is in the application cache then the access to it will be faster then if it is in the
device cache.
Applications and operating systems implement their own caching structures that use
the main memory in the host. Data is typically read from disk media and cached in
the main memory of the host, providing better response times should the same data
be required again.
At the controller and disk level, another level of cache is used to increase data
transfer to and from disk media. If the application requests a read from a disk drive,
the disk drive will transfer more data than was requested and store this in cache.
Applications that access data sequentially will benefit from this caching technique,
but applications that are random will not.
UC434S F.00
14 -41
Essentially, each read is converted into a larger read, usually the data from a full
drive rotation. If the data is not required by the application, then it is put into cache
only to be overwritten by the next read.
Disk caching affects performance in the following areas:
It reduces disk access (with cache hits).
It reduces the negative effects of the RAID overhead.
It assists in disk I/O request sorting and queuing.
Many disk controllers have a configurable cache memory. They provide read-only
caching, write caching, or a combination of read and write caching.
14 -42
UC434S F.00
Performance
Write-back caching
Write-back caching
CPU processing
Seek
Rotate
Xfr
CPU processing
Without write-back cache
With write-back cache
Cache
CPU processing data CPU processing
Seek
Time
Saved
Rotate Xfr
Done in background
Write-back caching enables the application to post the write request in the controller
cache and immediately respond with a completion status. The data is written to the
disk drives later.
Write cache is beneficial in high I/O capacity environments, where the I/O profile
includes random write requests. The write requests can be posted in the cache,
increasing the overall system performance if the workload is incremental, as shown in
the preceding diagram. When the write-back cache flushes, the incoming writes are
paused for a short time. If this happens when the workload is at a low point, then it
is correct. If the workload is constant, as in a backup restore, the write cannot be
completed and significant interruption can occur. Response times can grow
significantly.
For example, consider a database application with OLTP. In high-bandwidth write
environments, the write cache gets saturated easily and loses its effectiveness.
Write-back cache size

Write back cache size is not restrictive as long as the cache can be flushed to disk
faster than data arrives. More cache is better only if the current cache size is
inadequate.
UC434S F.00
14 -43
When cache is saturated, the response time is determined by the speed of the drives.
Most disk controllers allow cache memory upgrades to increase the cache size. Most
controller parameters can also be tuned.
When the cache is only partially used and not saturated, adding more cache
memory is a waste of resources. Performance does not improve.
14 -44
UC434S F.00
Performance
Write-back cache benefits

Write-back cache benefits
I/O per second
Six-disks RAID 5, 4KB random I/Os over 2GB at 20ms

450
400
350
300
250
200
150
100
50
0
WBC
No WBC
HP Storage 1
Storage 2
Storage 3
Storage 4
The preceding graph shows that performance increases significantly when write-back
cache is enabled in a write-intensive database environment. In addition to almost
eliminating response time, write-back cache also provides the following benefits:
Multiple writes to the same location are optimized because only the last update
is applied.
Multiple small sequential writes are combined into more efficient larger writes.
The read- and write-back penalty of RAID 5 can be eliminated for RAID
controllers that implement RAID 3/5
Write performance to log files is increased.
Protecting write-back cache

Because cache is an effective tool, it must be protected to prevent data loss.
HP StorageWorks controllers use:
Battery backup to preserve data in the event of power failure
Cache mirroring that is configurable with two controllers and protects against
hardware failure
UC434S F.00
14 -45
Cache coherency in dual controller configurations

Dual controller configurations present additional processing overhead on the array
controllerscache coherency. When a write request is posted to the active controller,
this controller must ensure that the request is also posted to the write cache of the
other controller before returning the completion status to the host.
The speed at which both caches are updated depends on the speed of the bridge
that connects the two array controllers.
14 -46
UC434S F.00
Performance
Effects of cache
Effects of cache
100
Response time (ms)
90
80
No cache
70
Read-ahead
cache
60
50
40
Write-back
cache
30
20
10
0
200
400
600
800
1000
Requests per second
1200
1400
The preceding graph summarizes the effects of read-ahead and write-back cache on
performance:
No cache With sufficient requests (enough load), response time is limited to

the speed of the disks.
Read-ahead cache Read performance improves with sequential data access.
Write-back cache Performance improves until the cache is saturated and then
writes occur at device speed.
Read-ahead caching
Read caching is used in two ways:
Read-ahead buffers These buffers are helpful during sequential read access.
When the disk controller detects sequential read patterns, it reads anticipated
information before the application requests it. This type of cache is called readahead cache.
Memory holding reusable information Any valid data located in the cache is
reused for new read requests. This type of cache is called most recently used
read cache.
Read-ahead cache is beneficial in high-bandwidth application environments such as

data marts, data warehouses, and backup servers. It improves performance during
any sequential read operations.
UC434S F.00
14 -47
However, if the application maintains its own cache in the system memory, the
chances of reusing information stored in the disk controller cache are minimal. The
data is more likely to be available from the application cache than from the disk
controller cache.
Cache is not always a performance booster. Read-ahead cache that experiences a
high incidence of cache misses can hurt performance. The preceding graph shows
the performance impact of read-ahead cache for theoretical HP StorageWorks
storage systems.
Performance for the HP StorageWorks systems remains the same in both cases as a
result of optimized system design. Cache performance and efficiency are attributed
largely to the design of the array controller. Although some vendors promote larger
caches as advantageous, performance depends on several factors, including
environment and cache design. Some controllers use adaptive read-ahead cache that
only reads ahead sequential I/O requests and does not affect non-sequential I/O.
14 -48
UC434S F.00
Performance
Application effects on performance

Application effects on performance
Two variables
I/O request patterns
Workload measured by
Number of I/O requests
Size of I/O requests
Applications
Bandwidth-intensive
I/O request-intensive
Not both
Data retrieval complexity

Number and types of operations performed on data
Comparisons and data summations increase complexity
Profiling is gaining an understanding of the users environment and the expectations

placed on the application. This information is essential to implementing an optimum
disk configuration.
HP makes these recommendations for database block size:
If performance is most critical during applications that primarily access the

database in a random fashion (small reads or updates scattered throughout the
database), use a block size of 2,048KB.
If most applications are accessing the database sequentially when performance

is most critical, use a block size of 16,384KB.
If you have a mix of random and sequential access during the most critical time,
use a block size of either 4,096KB or 8192KB depending on the ratio of random
to sequential access.
Because the transaction log is always configured with drive mirroring in large
systems, transaction log archiving allows the information to be migrated to less costly
parity-based schemes during periods of reduced activity.
UC434S F.00
14 -49
Environment profiling
Environment profiling
Database block size recommendations
If performance is most critical during applications that primarily
access the database in a random fashion; use a block size of
2KB
If most applications are accessing the database sequentially
when performance is most critical, use a block size of 16KB
If you have a split of random and sequential access during the
most critical time, use a block size of either 4KB or 8KB
The performance required by the average email user determines the storage design.
The average load is multiplied by the number of users to find the storage
requirement. Conversely, the capabilities of an existing system can determine the
maximum number of users.
To calculate the average I/O per user in an Exchange environment, the PERFMON
object's disk-transfers-per-second value is divided by the number of active
connections. The storage capacity calculated from the average I/O needs an
additional safety factor to maintain performance during peak periods. In practice,
the maximum number of users is less than the calculated value when:
Users increase the size of their mailboxes.
Services such as antivirus scanners or content indexers are added to the Exchange
server.
A medium-sized user profile provides a 60 MB mailbox, and a large profile provides
a 100 MB mailbox. Larger mailboxes affect both storage sizing and performance,
and are disproportionately more difficult for Exchange to manage.
14 -50
UC434S F.00
Performance
Large sequential read environment

Large sequential read environment
Performance gain in a given situation depends on the
I/O profile
I/O size
Ratio of read to write requests
Frequency
RAID level
9GB,10000
10000 RPM
9GB,
RPM
4.3GB,10000
10000 RPM
4.3GB,
RPM
9GB,7200
7200 RPM
9GB,
RPM
4.3GB, 7200
RPMRPM
4.3GB,
7200
10
15
20
512KB Sequential
sequential
Read,
read, single
Single stream
Stream
9GB, 10K
512KB Sequential
sequential
Read,
read, single
Single stream
Stream
4GB, 10K
512KB Sequential
sequential
Read,
Single stream
Stream
read, single
9GB, 7K
512KB Sequential
sequential
Read,
Single stream
Stream
read, single
4GB, 7K
MB/s
The preceding chart compares the performance of 7,200-rpm and 10,000-rpm drives
in a large sequential I/O read environment. This environment is representative of
video editing or streaming. In this case, performance gains are as much as 70% at
full capacity.
The performance gain in a given situation depends on the:
I/O profile
I/O size
Ratio of read to write requests
Frequency of reads/writes
RAID level
Faster drive technology does not automatically yield better performance. You must
analyze your system to determine where the most time is being spent.
The general rule to use is that for an application that requires high MB/s, choose
disk drives that have the highest rotational speeds and the highest data density. For
an application that requires high I/O per second, maximize the number of disks you
use.
UC434S F.00
14 -51
Server Application
Server Application
Application
Processor
Memory
Storage
Network
File Server
Light
Medium
Active
Active
Database Server
Active
Active
Active
Light
Web Server
Medium
Active
Active
Active
Domain controller Light
Active
Medium Medium
E-mail Server
Active
Active
Medium
Medium
Database Servers
SQL Server read/write sensitive
Oracle Server mixed profile, read/write dependent
Exchange Server random read/write I/O excl. logs
Note that the table above details typical examples of each server type. Notice that
Memory and Storage are almost always Active. These occur at either end of the
I/O path and, as such, there are direct performance consequences if there are
bottlenecks in these places.
Databases, mail and messaging

The fundamental I/O profile used to access the disk By examining different database
applications and database functions. That is, you can determine whether a particular
data set will be accessed either sequentially or randomly. The key high-level
application types include transaction processing, decision support, and batch
processing systems.
In Oracle databases, the database block size is defined during the installation, but
2KB is common for Online Transaction Processing (OLTP) applications. Microsoft SQL
Server 7.0 and SQL Server 2000 have a fixed 8KB block size.
OLTP applications frequently use the transaction log. Log writes are always performed
synchronously; each write must complete before the next write is issued. OLTP
applications generally do not require significant amounts of temporary storage.
14 -52
UC434S F.00
Performance
File System/ Database: file systems map continuous space (a file) onto the underlying
storage block units. In this respect database systems are no different than a file
system, and should be treated as a file system until a definite distinction has to be
made. Instead of mapping files, database systems map database tables onto
storage blocks. They must perform all the customary file system operations such as
open, read, write and backups.
Note: You must specify the DB_BLOCK_SIZE parameter before creating the database
because a change in DB_BLOCK_SIZE requires the database to be rebuilt.
Depending on the application, the value of DB_BLOCK_SIZE should be between 2KB
and 16KB.
SQL, Oracle, and Exchange share similar I/O profiles. The database and
information stores consist of random I/O with a high percentage of reads. Read
performance is crucial. Writes occur asynchronously and have little impact on users.
All multithreaded asynchronous write functions benefit from RAID.
The log areas consist of sequentially accessed data and should be physically
separated from the random I/O. In all three applications, this is a single-threaded,
low-queue-depth environment that does not benefit from RAID I/O distribution.
However, RAID 1 is usually implemented to protect crucial data. The speed of these
three applications depends largely on the speed at which requests are committed to
the log file. Log files with write-back cache enabled improve application
performance.
Note: Applications that issue multiple I/Os at a time benefit more from RAID than
environments where one I/O is issued at a time. Applications that do not issue more
than one I/O at a time do not benefit from RAID I/O distribution.
SQL Server 2000

SQL Server 2000 database files can be characterized on the application as:
Random access
8KB I/O size (2KB for SQL 6.5)
64KB I/O size for backup, table, and index scans
Typically 50% reads operations:
Asynchronous write operation (also known as the lazy writer):
Flushed in intervals
Write response time is not critical
Multithreaded
Queues build from asynchronous I/O
RAID improves performance
UC434S F.00
14 -53
SQL log files can be characterized as:
8 to 60KB I/O size (2KB for SQL 6.5)
100% synchronous sequential write operations
Single threaded, with no benefit from RAID
The peak performance on the server translates into one I/O per second per user.
From the previous tables, you can see that there are several solutions to the 180GB
information storage requirement with the associated number of available I/Os per
second to the application.
To determine if drive performance can be improved, review the system components.
The HBA can process nearly 10 times the I/O rate required without saturating, and
the switch port nearly 30 times. The controller is close to saturating and the workload
is random. The bottleneck seems to be caused by the disk physical characteristics
that are dominating the processing time.
Oracle 8 Server
Oracle 8 Server for Windows 2000 is a 32-bit application that is implemented on
Windows 2000 as a single process, multithreaded architecture.
Each Oracle 8 Server instance consists of:
A single Windows 2000 process with multiple Oracle 8 server threads
A System Global Area (SGA)
A Process Global Area (PGA)
A redo log
Control and configuration files
Typically, an Oracle 8 OLTP system can be characterized as many users acting on a

small subset of data throughout the database. The resulting I/O profile includes
heavy random reads and writes across the data and index files, while transaction
logs receive a heavy stream of sequential write operations of 2KB or less.
In this case, it is important to monitor the I/O rate on the disk subsystem and ensure
that individual disks are not overloaded with random I/Os.
Oracle 8 Server database files can be characterized on the application as:
14 -54
8KB I/O size (configurable up to 32KB for data warehousing)
Typically 30 to 60% read operations:
Asynchronous write operation
Write response time is not critical
Multithreaded
With queues
Improved performance with RAID

UC434S F.00
Performance
Oracle log files can be characterized as:
8KB I/O size
Because HP disk subsystems provide asynchronous input/output, Oracle8i Server

requires only one Database Writer thread and one Log Writer thread.
Exchange Server
The profile for Exchange Server can be characterized as:
4KB I/O size
30 to 50% read operations
response time is important
Asynchronous write operations
Flushed in intervals of 30 seconds
Response time is not critical
Multithreaded
With queues above 64
Improved performance with RAID
The log area can be characterized as:
4KB I/O size
Use the Microsoft Diskpart.exe utility to align the sectors of all Exchange LUNs
prior to formatting. Microsoft provides the diskpart.exe utility as part of
Windows 2003 Service Pack 1 support tools
UC434S F.00
14 -55
Reducing service time
Replace drives
Increase the number of drives
Reducing usage
Distribute the workload over more drives by using a drive array
and RAID technology
Shift the workload to another device if the application permits
Bypass the device with cache
14 -56
UC434S F.00
Performance
Comparing VRAID1 and VRAID5

Comparing VRAID1 and VRAID5
VRAID1 vs VRAID5 - IO Throughput
VRAID1 DBs
16000
VRAID5 DBs
VRAID1 Logs
VRAID5 Logs
14000
Disk Transfers/sec
12000
10000
8000
6000
4000
2000
0
9
12
15
Jetstress threads
18
21
When deploying VRAID5 LUNs, there is a performance penalty for write intensive applications because of the additional cost
of calculating and writing out the parity bit. The graph indicates that, as expected, I/O throughput is higher when the
database LUN is configured with VRAID1.
The VRAID comparison testing analyzed the performance results of VRAID1 as

compared to VRAID5 on a Exchange Server database using virtual disks. With any
implementation of a VRAID level, 1 or 5, there is a subsequent performance penalty
to implement the redundancy, specifically in write requests to the disk. For VRAID1,
two writes are required for a single write request from Exchange Server to write out
both the data block and the corresponding mirrored block for redundancy.
With VRAID5, there is a greater performance penalty of up to four additional disk
transactions to complete a single write request. For each write request, the following
sequence occurs:
Read the original data and parity block (two requests)
Calculate the new parity block
Write the new data and parity block (two requests)
The HP StorageWorks Enterprise Virtual Array 8000 (EVA8000) utilizes cache

optimization and write-gathering at the controller level to minimize the performance
penalty of VRAID5 writes. With write-gathering, multiple write operations are
grouped together to minimize the performance penalty of the parity update that
would occur for each individual write request to disk. With multiple writes grouped
together, the parity block need only be updated once. However, there is still a
performance penalty for VRAID5 compared to VRAID1 database and log virtual
disks.
UC434S F.00
14 -57
For both reliability and performance, the recommendation for an Exchange Server
installation is to isolate the database and transaction logs. For performance
considerations, HP recommends creating the largest possible disk group for the
database I/O streams and isolating the transaction log I/O streams on a separate
disk group.
As a quick background advisory, applications that utilize EVA VRAID disks might
experience a write performance penalty with the default Windows 2003 primary
disk partition alignment. Windows 2003 uses the first 63 sectors for volume
information before the start of the first partition, causing the first partition to start on
the last sector of the first track. Exchange Server 2003 writes out data in 4,000
chunks so every eighth I/O will cross a track boundary, resulting in additional
latency on the I/O request. Using the DiskPar utility before formatting the drive, the
alignment can be set so that the first partition begins with a sector offset alignment of
64, rather than the default 63, which causes the first partition to begin on a new
track without incurring any track overlapping.
14 -58
UC434S F.00
Performance
Safe IOPs calculator for production disk group
To use the spreadsheet:
Input the number of drives in the disk group.
Input the percent of read.
Look at the drive type and VRAID type to determine the total
number of host IOPS the disk group can handle and deliver
acceptable latencies.
Rules for sizing load are contained in this spreadsheet and are implemented through
the following variables:
Random I/O Performance is linear with the number of drives

added to a disk group, and is dependent on:
Drive speed 10K rpm or 15K rpm
Percent of read requests versus write requests
Write penalty for VRAID1, VRAID5, and VRAID6
Percentage of VRAID1 and VRAID5
Sequential I/O Performance is not linear with the number of

drives and typically is limited by components of the EVA
(processor, bus, and so on)
UC434S F.00
Each physical disk can read 10MB/s regardless of

VRAID.
14 -59
For writes, VRAID1 can do 10MB/s divided by a VRAID

penalty of 2.
For writes, VRAID5 can do 10MB/s divided by a VRAID

penalty of 4/5.
This spreadsheet puts all the rules into one place and allows you to calculate and use
baseline numbers for performance data.
14 -60
UC434S F.00
Performance
Safe IOPs calculator Microsoft version
Above is the SAFE IOPs calculator available through Microsoft site and is
available to customers. As with the HP internal version, you input the number of
drives and the read percent to get a disk group Safe IOPs number
UC434S F.00
14 -61
EVAPerf
EVAPerf
To capture the necessary statistics for analysis, Windows Performance Monitor was
utilized along with the EVAPerf add-in that enables monitoring of specific EVA
subsystem counters.
Windows Performance Monitor counters

Windows Performance Monitor (Perfmon) is an MMC snap-in that enables
monitoring of the utilization of operating system resources such as CPU, memory, and
disk. The counters that are discussed in this white paper are described in the
following sections.
Physical disk counters

The physical disk counters keep track of information pertaining to each instance of a
disk presented to the server. There is an instance of these counters for each physical
disk presented to the Windows operating system on the server.
14 -62
Disk Transfers/secThe rate of read and write operations on the disk
Disk Bytes/secThe rate bytes are transferred to or from the disk during write or
read operations
Disk Writes/secThe rate of write operations on the disk
Disk Reads/secThe rate of read operations on the disk
Avg. Disk sec/writeThe average time, in seconds, of a write of data to the

disk
UC434S F.00
Performance
Avg. Disk sec/readThe average time, in seconds, of a read of data to the disk
Avg. Disk sec/transferThe average time, in seconds, of the average disk

transfer
EVAPerf counters
The EVAPerf utility is an add-in to the Windows Performance Monitor for monitoring
of the EVA subsystem.
EVA physical disk counters

The physical disk counters keep track of information on each physical disk on the
system. There is no information relating these disks to a specific disk group, nor is the
activity broken out into the underlying cause of the I/O, such as host driven, cache
flushes, read-ahead, leveling, and snapshot activity.
There is one instance of these counters for each physical disk on the EVA. Each disk
is uniquely identified by a four-digit hexadecimal number. This number is an internal
representation of the disk used by the EVA known as a noid and has no
relationship to the shelf or bay where this disk resides.
Drive latencyThis counter tracks the time between when a data transfer
command is sent to a disk and when command completion is returned from the
disk. This time, which is measured in microseconds, is not broken into read and
writes latencies but is simply a command processing time. Note that
completion of a disk command does not necessarily imply host I/O completion
because the I/O to a specific disk might be only a part of a larger I/O
operation.
Drive Queue DepthThis counter tracks the total number of requests that have
been sent to the drive but not yet completed. It is incremented whenever a
command is sent to the disk and decremented whenever a command completes.
Read RPSThis counter tracks the number of read requests that have been sent
to the disk drive. Because this counter is updated once per second, it translates
directly into the read requests per second.
Write RPSThis counter tracks the number of write requests that have been sent
to the disk drive. Because this counter is updated once per second, it translates
directly into the write requests per second.
EVA VDisk counters

The VDisk object tracks performance for each virtual disk (LUN) on the EVA. It is
similar to the physical disk object, but it tracks virtual LUNs instead.
There is one instance of these counters for each virtual disk on the EVA. Each VDisk
is uniquely identified by a four-digit hexadecimal number. This number is an internal
representation of the LUN used by the EVA known as a noid and has no
relationship to the LUN number.
UC434S F.00
14 -63
Read Hit LatencyThis counter tracks the time taken from when a host read
request is received until such time as that request has been satisfied from the
EVA cache memory. The time, which is measured in microseconds, only applies
to read commands that are satisfied from read cache. If the read command is a
cache miss, the time is not tabulated here (see Read Miss Latency). Note that
this value includes not only the latency from cache hits generated from random
access activity, but also the latency associated with a cache hit as a result of a
prefetch operation generated by a sequential read data stream.
Read Miss LatencyThis counter tracks the time taken from when a host read
request is received until such time as that request has been satisfied from the
physical disks. The time, which is measured in microseconds, only applies to
read commands where the data is not in read cache and must be read from
disk. If the read command results in the data being read from cache, the time is
not tabulated here.
Write LatencyThis counter tracks the time, measured in microseconds, between

when a write command is received from a host and when command completion
is returned.
Write RPSThis counter tracks the total number of write requests to a virtual disk
that were received from all hosts. Because this data is updated once per second,
it translates directly into write requests per second.
EVA storage cell counters

The storage cell object tracks information that is related to the overall storage system.
It is a quick roll-up of several of the important metrics associated with overall EVA
performance. There is only a single instance for these counters; this single instance
represents the sum total of both controllers.
14 -64
Total host KBSThis counter tracks the total KB that has been read and written
by all hosts connected to the EVA. Because this information is updated once per
second, it translates directly into the total KB per second that the EVA is
processing. Note that this is the sum of both read and write data.
Total host RPSThis counter tracks the total number of I/O requests that have
been issued by all hosts connected to the EVA. Because this information is
updated once per second, it translates directly into the total requests per second
that the EVA is processing. Note that this is the sum of both read and write
requests.
UC434S F.00
Performance
End to End monitoring

End to End monitoring
Add Monitors here
perfaddeemonitor [slotnumber/]portnumber sourceID destID
monitor end-to-end performance using the perfMonitorShow
End-to-end performance monitoring looks at traffic on SID/DID pairs in any direction. That is,
even if the SID is for a remote device, the traffic is monitored in both directions (the Tx/Rx
counters are reversed).
End-to-end performance monitoring counts the number of words in Fibre Channel

frames for a specified Source ID (SID) and Destination ID (DID) pair. An end-to-end
performance monitor includes these counts:
RX_COUNT - words in frames received at the port
TX_COUNT - words in frames transmitted from the port
To enable end-to-end performance monitoring, you must configure an end-to-end
monitor on a port, specifying the SID-DID pair (in hexadecimal). The monitor counts
only those frames with matching SID and DID.
Adding end-to-end monitors
1. Connect to the switch and log in as admin.
2. Enter the following command:
perfaddeemonitor [slotnumber/]portnumber sourceID destID
monitor end-to-end performance using the perfMonitorShow command
UC434S F.00
14 -65
Top talker
Top talker
Top Talker supports two modes:
port mode and fabric mode
Adding a Top Talker monitor on to an F_Port
To monitor an incoming port 5:
perfttmon --add ingress 5
To monitor the outgoing traffic on port 5:
perfttmon --add egress 5
To delete the monitor on port:
perfttmon --delete 7
allows for the configuration of QoS attributes to assign effective priority
Top Talkers
The Top Talkers feature is part of the licensed Advanced Performance Monitoring
The Top Talkers feature provides real-time information about the bandwidth being
consumed on a specific port; it identifies the SID/DID pairs that consume the most
bandwidth. This then allows for the configuration of QoS attributes to assign
effective priority.
Top Talker can be installed only on switches that run Fabric OS v6.0.0 or later, and
is not supported on the B-Series 4/16 (Brocade 200E).
Top Talker supports two modes: port mode and fabric mode.
Port mode Top Talker, this allows for Top Talker to be installed on to an F_Port,
allowing for the measurement of bandwidth used by this port to different
destinations.
In Fabric mode Top Talker monitors are installed on every E_Port measuring the
data rate of all the incoming data flow to the E_Port, allowing for the
determination of the highest bandwidth device.
Top Talker monitors can be configured as Port Mode or Fabric Mode but not both.
End-to-end monitors provide counter statistics for traffic flowing between a given
SID-DID pair. Top Talker monitors identify all possible SID-DID flow combinations that
are possible on a given port and provides a sorted output of the top talking flows.
14 -66
UC434S F.00
Performance
If the number of flows exceeds the hardware resources, existing end-to-end monitors
fail to get real time data for all of them; however, Top Talker monitors can monitor all
flows for a given port (E_Port or F_Port).
Adding a Top Talker monitor on to an F_Port
To monitor an incoming port 5:

perfttmon --add ingress 5
To monitor the outgoing traffic on port 5:

perfttmon --add egress 5
To delete the monitor on port:

perfttmon --delete 7
Limitations of Top Talker monitors

Be aware of the following when using Top Talker monitors:
Top Talker monitors cannot detect transient surges in traffic through a given flow.
Top Talker monitor cannot be installed on a mirrored port.
Top Talker can monitor only 10,000 flows at a time.
Top Talker is not supported on VE_Ports, EX_Ports, and VEX_Ports.
UC434S F.00
14 -67
Lab activity
Lab
activity
1.Module 14, Lab 1 Performance Testing Your
SAN Volumes
2.Module 14, Lab 2 - B-series
Trunking
43
14 -68
uc434s c.01
UC434S F.00

Uc434s.f.00 Stu - Part2 PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Uc434s.f.00 Stu - Part2 PDF

Uploaded by

Copyright:

Available Formats

Accelerated SAN Essentials

Accelerated SAN Essentials

Copyright 2010 Hewlett-Packard Development Company, L.P.

2010 Hewlett-Packard Development Company, L.P.

Accelerated SAN Essentials

Module 9 - SAN extension

2010 Hewlett-Packard Development Company, L.P.

Module 11 - SAN Management

2010 Hewlett-Packard Development Company, L.P.

Accelerated SAN Essentials

Database Viewer ......................................................................................11 - 31

Module 12 - SAN Security

Module 13 - Data protection

2010 Hewlett-Packard Development Company, L.P.

Not all data are equal ............................................................................... 13 - 4

2010 Hewlett-Packard Development Company, L.P.

Accelerated SAN Essentials

2010 Hewlett-Packard Development Company, L.P.

2010 Hewlett-Packard Development Company, L.P.

Accelerated SAN Essentials

Voice and video transmission

IP networking is based on design considerations different from those of storage

Addresses the difficulties of managing burgeoning storage resources.

Facilitates the storing, accessing, protection, and management of mission-critical

Provides protection of data by allowing redundant paths between host and

Enables remote mirroring solutions for disaster recovery.

Allows backups to be performed over the IP storage with minimal impact on

Allows storage to be consolidated, which reduces management complexity.

Centralized management of a consolidated storage pool can be more efficient than

2010 Hewlett-Packard Development Company, L.P.

Accelerated SAN Essentials

Understanding IP storage protocols

2010 Hewlett-Packard Development Company, L.P.

The following three IP storage encapsulation protocols have been defined:

Fibre Channel over TCP/IP

VE_ports and VEX_ports

Internet Fibre Channel Protocol

2010 Hewlett-Packard Development Company, L.P.

Accelerated SAN Essentials

Overview of iSCSI protocol

2010 Hewlett-Packard Development Company, L.P.

iSCSI maps SCSI onto a network

SCSI is a long-established protocol for connecting disks to computers. All

2010 Hewlett-Packard Development Company, L.P.

Accelerated SAN Essentials

Overview of iSCSI protocol

Overview of iSCSI protocol

The iSCSI protocol is a means of transporting of the SCSI

Why was it created?

2010 Hewlett-Packard Development Company, L.P.

2010 Hewlett-Packard Development Company, L.P.

Accelerated SAN Essentials

Universal storage access across an IP network regardless of the interconnect (for

Access to existing Fibre Channel SANs and a migration path to an IP

High-end storage and SAN expansion for iSCSI servers

Improved storage manageability and high availability of storage resources

A core SAN fabric that is IP-based

These environments can provide application support for:

Local and remote backup over an IP network.

Storage virtualization across a common pool of heterogeneous storage

Disaster recovery and high availability.

Mirroring across heterogeneous SANs.