Vug LPM

Power Systems ATS
Live Partition Mobility Update

Ron Barker
Power Advanced Technical Sales Support
Dallas, TX
2013 IBM Corporation
Power ATS
Agenda
Why you should be planning for partition mobility
What are your options? Which is best for you?
How to set up for LPM using vSCSI
How to set up for LPM using NPIV
Tips that will make setting it up easier
The mobility process start to finish
Questions?
Power ATS
The value of partition mobility
Flexibility: move workloads to servers that are more ideally suited

to your needs
Application requirements can change
The infrastructure should adapt quickly
Service levels must be maintained
Availability: move workloads off machines that are scheduled for

planned maintenance
Hardware migrations made easier if workloads can temporarily be run
elsewhere without interruption
Remove the chance of disruption when a system firmware upgrade or
concurrent hardware maintenance is planned
Power ATS
What options are there?
Use redundant systems and PowerHA, bringing one server down

and moving its workload to another
Additional hardware and software costs, plus configuration and testing
Schedule a maintenance window and bring down all partitions and

applications
Very disruptive; hard to get users to agree on when a maintenance
window should be scheduled
Use Live Partition Mobility to migrate a partition from one server to

another without an interruption in service
Works on any Power server with PowerVM Enterprise Edition, but
requires complete I/O virtualization
Power ATS
What are your options?
All storage virtualization technologies support migration

Virtual SCSI: One or a pair of Virtual I/O Servers supply the client
partition with generic hdisks virtualized from SAN storage owned by the
VIOS lpars
N-Port ID Virtualization (NPIV): VIO server HBAs act in pass-thru mode
to connect Virtual Fibre Channel adapters to the SAN environment,
where they are zoned and mapped to LUNs owned by each client
Shared Storage Pools: Cluster of up to 16 VIO servers share
management of a large pool (up to 512 TB) of SAN storage that is
carved into file-backed Logical Units and allocated as hdisks to clients
Power ATS
Readiness Checklist
PowerVM Enterprise Edition license on source and target
servers
Source and target servers are POWER6 or POWER7
If processor architectures are different, the mobile
partition must use compatibility mode
Check matrix for firmware level compatibility
http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/index.jsp?to
pic=/p7hc3/p7hc3firmwaresupportmatrix.htm
Both servers are HMC managed
Logical memory block size the same on source and target
If Active Memory Sharing is used, make sure a shared
memory pool exists on the target server
Power ATS
Readiness Checklist (contd)

If Active Memory Expansion is used, make sure its
supported on the destination server
If the mobile partition is suspend-resume capable, make sure
the target has a reserved storage pool greater than or equal
to 110 percent of the lpar size
If you are moving an IBM i mobile partition, verify that the
destination server supports the migration of IBM i mobile
partitions and the restricted I/O mode
Also, verify that the IBM i mobile partition is in the
restricted I/O mode
Power ATS

If using a shared processor pool with an entitlement below
0.1 processing units and greater than or equal to 0.05
(POWER7+), make sure the destination server also shares
that capability
If the mobile partition is using a virtual Ethernet adapter which
is using a virtual switch in the VEPA (Virtual Ethernet Port
Aggregator) mode, or the mobile partition is using a virtual
Ethernet adapter with a VSI profile, then verify that the
destination server also supports virtual server network (VSN)
The legacy virtual switch mode on Power is VEB, or
Virtual Ethernet Bridge, and is most common today
Power ATS
All VIO servers and clients must be on the same subnet

Virtual SCSI disks must be on shared LUNs (disks set to
no_reserve) *One exception mentioned later
LUNs using NPIV need to be mapped to both world wide
port names of each client Virtual Fibre Channel adapter
All I/O resources must be shared or virtualized prior to
migration dedicated devices need to be removed
Processors and memory may be dedicated or shared
SAN switch must support NPIV, at least the port to which the
VIOS physical adapter connects
Make sure the SAN switch software is at the latest
supported level
Power ATS
Setting up a VIOS and VIO clients using vSCSI

Size VIOS using Workload Estimator http://www912.ibm.com/estimator
Or use the rule-of-thumb: 1.0-2.0 processing units from the
shared pool, uncapped, with 2-4 GB of memory (4 GB
Shared Storage Pool)
1-10 Gbps Ethernet or Integrated Multi-function Card (in
promiscuous mode for Shared Ethernet Adapter usage)
One or more 8 Gbps dual-port Fibre Channel adapters
Current VIOS operating system, e.g., VIOS 2.2.2.2-FP26 SP
01
VIOS can boot from SAN, but the best practice is to boot
from internal disks
10
Power ATS
Setting up a VIOS and VIOC using vSCSI

Designate the VIOS as a Mover Service Partition (MSP)
Make sure no virtual adapter is Required
Enable Time reference on Settings tab, under Service and
Support
Zone and map storage from the SAN to the VIOS for
virtualization
When creating the VIOC. make sure Virtual Adapters are
listed as not required
Using the HMC, carefully map slot numbers for each Server
SCSI adapter (vhost) to the slot of a corresponding Client
SCSI adapter
Refer to IBM PowerVM Virtualization Introduction and Configuration for how-to
details:
http://w3.itso.ibm.com/abstracts/sg247940.html?Open
11
Power ATS
Setting up a VIOS and VIOC for NPIV

VIOS creation the same as for vSCSI, except the physical
adapter's world wide port name is not mapped to the client's
LUNs
VIOS HBA is mapped to the switch so it can perform its
pass-thru function for the client VFCs
Both world wide port names from each client VFC are zoned
and mapped to storage (one's active and one's standby)
There's a one-to-one relationship between a Server VFC
and a client VFC
A Server VFC is mapped to one of the ports on a physical
HBA (64 VFC hosts are supported on each HBA port); only
one client VFC supported per physical server port
12
Power ATS
Tips to make setup easier

The Hypervisor will automatically manage migration of CPU
and memory, but it must be available on the target server
The administrator should closely monitor the validation and
MSP pairing to make sure the desired VIO servers are used
and the correct MPIO mappings are maintained
Dedicated I/O adapters, if any, must be de-allocated before
migration, but available dedicated I/O adapters may
be re-added on the target after the migration
13
Power ATS

The reserve attribute on the physical storage that is used by
the mobile partition can be set as follows:
1. You can set the reserve policy attribute to no_reserve
2. You can set the reserve policy attribute to pr_shared
when the following products are at the following
versions:
HMC version 7 release 3.5.0, or later
VIOS version 2.1.2.0, or later
The physical adapters support the SCSI-3 Persistent

Reserve standard
The reserve attribute must be the same on the source and

destination VIOS partitions for successful Partition Mobility.
14
Power ATS

Where do you find the wwpns?
On the HMC, look at the client profile and view the
details of all client VFC adapters
From HMC command line, use the lssyscfg command
ssh hscroot@hmc.com 'lssyscfg -r prof -m <server> --filter
"lpar_names=mob79_redhat
-F lpar_name,name,virtual_fc_adapters
mob79_redhat,default,"""7/client/1/ec01_vio1/27/c050760003ec0048,c050
760003ec0049/0"",""8/client/2/ec02_vio2/27/c050760003ec0046,c0507
60003ec0047/0""
Boot a new client lpar in SMS to expose wwpns to the

switch, allowing the administrator to do zoning and
mapping
15
Power ATS
Both GUI and Command Line Interfaces
16
Power ATS
Command Line Interface -- migrlpar

Migrations can be scripted using the migrlpar command
Run migrlpar --help or man migrlpar to see the syntax (its
updated as features are added)
You can be very specific in selecting the source and

destination MSP servers, the mapping of virtual SCSI and
virtual FC servers and slots, and other configuration details
17
Power ATS
Command line interface migrlpar (contd)

When using virtual_fc_mappings and virtual_scsi_mappings
attribute pairings, enclose in double quotes preceded by a
backslash (\)
New migrlpar flags:
-- mpio [1|2] Require MPIO pathing from source be
duplicated on target; 1 (default) means require multipathing; 2 means to try, but allow regardless
-- vlanbridge [1|2] Require target to have VLAN access
to the same external network; 1 (default) means yes; 2
says to try but allow regardless
-- protectstorage [1|2] When migrating a suspended
partition, 1 requires the storage be activated for the
move to prevent accidental reassignment to another
partition; 2 means you assume responsibility for the
safety of the suspended partitions storage
Power ATS
Validation High Level

Capability and compatibility check
Resource Monitoring and Control (RMC) check
Partition readiness
System resource availability
Virtual adapter mapping
Operating system and application readiness check
19
Power ATS
Validation
Please Note: there are other validation steps, this

presentation simply shows some of the sequence.
System Properties support Partition Mobility

Inactive and Active Partition Mobility Capable = True
Mover Service Partitions on both Systems
VIO Servers with VASI device defined, and MSP enabled
20
Power ATS
Validation Output
Looking at a validation output screen can be confusing
because warnings are listed as well as errors
Warnings wont derail a migration, but errors do

Some pervasive issue such as incorrect mapping of virtual
adapters -- will show up multiple times, usually as the same
failure on a different object
21
Power ATS
Sample Validation Errors
This indicates SSH has not been set up between a local

and a remote HMC in order to do a Remote HMC migration
$ mkauthkeys -g --ip <HMC_ip_address> -user
<HMC_user_for_both HMCs> --passwd <password> -t rsa
22
Power ATS
Sample Validation Errors
This refers to RMC communication and probably can be

fixed by stopping and restarting rsct daemons on the
client partition
23
Power ATS
Migration
If validation passes, migration can begin
From this point, all state changes are rolled back if an error
occurs
Mobile
Partition
MSP
VASI
POWER Hypervisor
Source System
Network
Mobile
Partition
MSP
VASI
5
POWER Hypervisor
Target System
Partition State Transfer Flow
24
Power ATS
Migration Steps (1 of 6)
The HMC creates a shell partition on the destination system
The HMC configures the source and destination Mover
Service Partitions (MSP)
MSPs connect to PHYP thru the Virtual Asynchronous
Serial Interface (VASI)
The MSPs set up a private, full-duplex channel to transfer

partition state data
25
Power ATS
The HMC sends a Resource Monitoring and Control (RMC)
event to the mobile partition so it can prepare for migration
The HMC creates the virtual target devices and virtual SCSI
adapters in the destination MSP
You do NOT do this manually; it must happen under
Hypervisor control
The MSP on the source system starts sending the partition

state to the MSP on the destination server
26
Power ATS
The source MSP keeps copying memory pages to the target
in successive phases until modified pages have been
reduced to near zero
The MSP on the source instructs the PHYP to suspend the

mobile partition
The mobile partition confirms the suspension by suspending

threads
27
Power ATS
The source MSP copies the latest modified memory pages
and state data
Execution is resumed on the destination server and the

partition re-establishes the operating environment
The mobile partition recovers I/O on the destination server

and retries all uncompleted I/O operations that were going on
during the suspension
It also sends gratuitous ARP requests to all VLAN
adapters
28
Power ATS
When the destination server receives the last modified pages,
the migration is complete
In the final steps, all resources are returned to the source and
destination systems and the mobile partition is restored to its
fully functional state
The network channel between MSPs is closed

The VASI channel between MSP and PHYP is closed
VSCSI and VFC adapters on the source MSP are removed
29
Power ATS
The HMC informs the MSPs that the migration is complete
and all migration data can be removed from their memory
tables
The mobile partition and all its profiles are deleted from the
source server
You can now add dedicated adapters to the mobile partition

via DLPAR as needed, or put it in an LPAR workload group
30
Power ATS
Effects
Server properties
The affinity characteristics of the logical memory blocks may change
The maximum number of potential and installed physical processors
may change
The L1 and/or L2 cache size and association may change
This is not a functional issue, but may affect performance
characteristics
Console
Any active (serial) console sessions will be lost when the partition is
migrated
Console sessions must be re-established on the target system by the
user after migration
31
Power ATS
Effects
Network
A temporary network outage of seconds is expected to occur as part
of suspending the partition
Temporary network outages may be visible to application clients,
but it is assumed that these are inherently recoverable
VSCSI Server and Server Virtual Fibre Channel Adapters

Adapters that are configured with the remote partition set to the
migrating partition will be removed
Adapters that are configured to allow any partition to connect will
be left configured after the migration
Any I/O operations that were in progress at time of the migration will
be retried once the partition is resumed
As long as unused virtual slots exist on the target VIO server, the
necessary controllers and target devices will be automatically created
32
Power ATS
Effects
Error logs
When a partition migrates, all of the error messages that the partition
received will appear on the target system
Error logs contain the machine type, model and serial number so it is
possible to correlate the error with the system that detected it
Partition time
When a partition is migrated the Time of Day and timebase values of
the partition are migrated.
The Time of Day of the partition is recalculated ensuring partition
timebase value increases monotonically and accounting for any
delays in migration.
33
Power ATS
If slot numbers cannot be maintained
or server/HMC capabilities differ
34
Power ATS
Make selections carefully for redundancy
The default
selected by PHYP
was ce51: vio1,
which was found
first searching by
lowest slot and then
partition name;
ec01 and ec02
were chosen
manually
35
Power ATS
Watch which Mover Service Partition was selected
Notice that the

MSPs may not be
ordered the way
you want or
expect; they
apparently are
sorted by partition
name
36
Power ATS
You can make selection manually
37
Power ATS
Review Validation Errors/Warnings
This appears when different

HMC capabilities or server
capabilities exist on the two
servers
38
Power ATS
Questions?
Power ATS
Helpful References
IBM PowerVM Live Partition Mobility, SG24-7460
PowerVM Migration Physical to Virtual Storage, SG24-7825
IBM PowerVM Virtualization Introduction and Configuration, SG247940-05
IBM PowerVM Virtualization Managing and Monitoring, SG247590-04
IBM Redbooks home page:

http://www.redbooks.ibm.com

Vug LPM

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vug LPM

Uploaded by

Copyright:

Available Formats

Power Systems ATS

Live Partition Mobility Update

2013 IBM Corporation

2013 IBM Corporation

The value of partition mobility

Flexibility: move workloads to servers that are more ideally suited

Availability: move workloads off machines that are scheduled for

2013 IBM Corporation

What options are there?

Use redundant systems and PowerHA, bringing one server down

Schedule a maintenance window and bring down all partitions and

Use Live Partition Mobility to migrate a partition from one server to

2013 IBM Corporation

What are your options?

All storage virtualization technologies support migration

2013 IBM Corporation

2013 IBM Corporation

Readiness Checklist (contd)

2013 IBM Corporation

Readiness Checklist (contd)

2013 IBM Corporation

Readiness Checklist (contd)

All VIO servers and clients must be on the same subnet

2013 IBM Corporation

Setting up a VIOS and VIO clients using vSCSI

2013 IBM Corporation

Setting up a VIOS and VIOC using vSCSI

2013 IBM Corporation

Setting up a VIOS and VIOC for NPIV

2013 IBM Corporation

Tips to make setup easier

be re-added on the target after the migration

2013 IBM Corporation

Tips to make setup easier

HMC version 7 release 3.5.0, or later

VIOS version 2.1.2.0, or later

The physical adapters support the SCSI-3 Persistent

The reserve attribute must be the same on the source and

2013 IBM Corporation

Tips to make setup easier

Boot a new client lpar in SMS to expose wwpns to the

2013 IBM Corporation

Both GUI and Command Line Interfaces

2013 IBM Corporation

Command Line Interface -- migrlpar

You can be very specific in selecting the source and

2013 IBM Corporation

Command line interface migrlpar (contd)

2013 IBM Corporation

Validation High Level

2013 IBM Corporation

Please Note: there are other validation steps, this

System Properties support Partition Mobility

2013 IBM Corporation

Warnings wont derail a migration, but errors do

2013 IBM Corporation

Sample Validation Errors

This indicates SSH has not been set up between a local

2013 IBM Corporation

Sample Validation Errors

This refers to RMC communication and probably can be

2013 IBM Corporation