Professional Documents
Culture Documents
SOS05
Alexander Warmuth
Objectives
Agenda
A B
Storage Server
Device Mapper
Block
multipath
Devices
multipathd
SCSI
HBA HBA
Hardware
0 1
SAN
/dev/disk/by-id/...
A B
/by-path/...
Storage Server /by-uuid/...
AVT on
– full LUN access on both controllers
– LUN automatically transferred to accessing controller
– Not for cluster solutions with shared access to volumes
AVT off, RDAC Mode
– SCSI Inquiry, Test Unit Ready, Mode Sense / Select,
Reserve / Release, Read Capacity possible
– No Read / Write access through alternate controller
SAN
A B
Storage Server
Block
Devices
RDAC Multipathing Driver
mppVhba
RDAC upper layer
Block
Devices – Inhibits mapping of block devices
SCSI
– No device scan
mppUpper RDAC virtual HBA
HBA
0
Hardware
HBA
1
– New block devices
– Regular integration
SAN
– Device Mapper on top possible
A B
Storage Server
DM-MP Development
Block
Devices
Hardware Handler in SCSI layer
Device Mapper
Inhibit device scan
Block
Devices
Multipathing based on UUID
SCSI
RDAC RDAC No read access neccessary
HBA HBA
Hardware
0 1
SAN
A B
Storage Server
SAN
Change defaults
A B defaults {
Storage Server
failback immediate
no_path_retry "queue"
}
Controlling DM-MP
x3650-lab-10:~ # multipathd -k
multipathd> help
multipath-tools v0.4.8 (08/02, 2007)
CLI commands reference:
list|show paths
list|show maps|multipaths
add path $path
list|show maps|multipaths status
remove|del path $path
list|show maps|multipaths stats $map
add map|multipath
list|show maps|multipaths topology
remove|del map|multipath $map
list|show topology
switch|switchgroup map|multipath $map group $group
list|show map|multipath $map topology
reconfigure
list|show config
suspend map|multipath $map
list|show blacklist
resume map|multipath $map
list|show devices
reinstate path $path
fail path $path
multipathd>
Block
Devices
Initial RAM
SCSI
Disk
HBA HBA
Hardware
0 1
SAN
A B
Storage Server
SAN
Boot from multipath
– Some boot loaders not yet DM-MP
A B aware
Storage Server – BIOS problems still exist
SAN Boot,Vendor:
kernel: DM-MP IBM andModel:
DS4000
1814 FAStT Rev: 0916
kernel:
Block
Type: Direct-Access Device scans lead to SCSI
ANSI errors
SCSI revision: 05
Devices
kernel: SCSI device sde: 104857600 512-byte hdwr sectors (53687 MB)
– Long
kernel: sde: Write Protect is off installation time
Device Mapper
kernel: sde: Mode Sense: 77 00 10 08
– Long boot time
kernel: SCSI device sde: drive cache: write back w/ FUA
Block
kernel: – Boot
SCSI device sde: 104857600
Devices loader
512-byte configuration
hdwr sectors (53687problems
MB)
kernel: sde: Write Protect is off
SCSI
kernel: sde: Mode Sense: 77 00 10 08 Easier with AVT on
kernel: SCSI device sde: drive cache: write back w/ FUA
HBA HBA
Hardware
0
kernel: 1
sde:end_request: I/O error, dev sde, sector 0
kernel: Buffer I/O error on device sde, logical block 0
kernel:
SANend_request: I/O error, dev sde, sector 0
kernel: Buffer I/O error on device sde, logical block 0
kernel: end_request: I/O error, dev sde, sector 0
A
kernel: BufferB I/O error on device sde, logical block 0
Storage Server
kernel: unable to read partition table
kernel: sd 3:0:0:1: Attached scsi disk sde
kernel: sd 3:0:0:1: Attached scsi generic sg12 type 0
N-Port ID Virtualization
What is NPIV?
HBA “multiplication”
– Virtual adapters presented to the operating system
– Virtual N-ports presented to Fabric
• Dedicated WWPN
• Appears just as physical port
HBA and switch must be NPIV capable
Used in virtualized systems
– VMWare
– Linux on System z and VM
– XEN integration in implementation
?
ar
Physical volume
ye
File
DomU3
Logical Volume
Dom0 DomU2
st
Image file
la LVol DomU1
er
XEN Hypervisor
b
em
em
R
XEN Hypervisor
N-Port
N-Port
Switch
N-Port
N-Port
Storage Server
N-Port
Switch
N-Port
N-Port
Storage Server
XEN Hypervisor
Assigned to VM
N-Port V-Port V-Port
Volume Mapping to V-Port
N-Port V-Port V-Port
Switch
N-Port
N-Port
Storage Server
N-Port
N-Port
Storage Server
HBA Information
?
ar
Linux can't use additional capacity on-the-fly
ye
– Minimum interruption: unmount file system
– Reboot usually not necessary
st
la
LVM: proven method to add capacity online
er
– Add new disk
Linux host
b
– Expand volume group HBA 0 HBA 1
+ Cap
Agenda
Summary
Sources of Information
More Questions?
Contact: warmuth@de.ibm.com
The information could include technical inaccuracies or typographical errors. Changes are periodically made to the
information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in
the product(s) and/or the program(s) described in this publication at any time without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve
as an endorsement of those Web sites. The
materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.
IBM makes no representation or warranty regarding third-party products or services including those designated as ServerProven,
ClusterProven or BladeCenter Interoperability Program products. Support for these third-party (non-IBM) products is provided by non-IBM
Manufacturers.
IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not
give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive,
Armonk, NY 10504-1785 USA.