July 20, 2004

Managing Logical Partition States

John Hughes
Advanced Technical Support
Washington Systems Center
jjhughes@us.ibm.com

1.0 Information Sources

This paper is primarily based on a support center alert MA052504B Managing LPAR
Partition States / Best Practices to Share with Your Customers to Prevent and Clear
Observed High Channel Utilizations.
It also includes information developed and provided by zSeries Product Engineering
and zSeries Channel Development.

2.0 Machine Types and Models

MACHINE TYPES: 9672, 2064, 2066, 2084, and 2086
MODELS AFFECTED: All Models at Drivers 26, 3G, 52, and 55

3.0 Introduction
It is important to be aware of the state of the zSeries processor image given the I/O con-
figurations and operating environments seen today.
These environments include:
• Multiple logical partition images sharing the same I/O subsystems
• New I/O subsystems which support more device transition functions
• Operating environments including XCF, PAV, Flash Copy, synchronous PPRC, and
other I/O subsystem functions.

1 of 11
 Introduction

Without the awareness of the state of a particular partition image and potential delay of
operator response to that state, certain states of the image may cause an adverse impact
to channel performance. Specifically, clients have observed high channel utilization on
a production image when a test image on the same processor, sharing the same channels,
enters a disabled waitstate. For example, the following is taken from RETAIN Tip
H181594:

Description:

A problem has been seen where FICON channel path activity is running consistently
high as observed by either the software System Activity Display(SAD) on the zSeries
CPC or, through software measurements such as Resource Measurement Facility
(RMF) in the Channel Path Activity report. In conjunction with the high channel path
activity, clients may also observe some type of performance slowdown in their FICON
channel utilization.

One possible cause for these problems can be the combination of asynchronous I/O
interrupts being presented to an image which is in a disable waitstate. With no operat-
ing processors, the channel is not able to clear interrupt conditions. This results in
excessive channel processing with each attempt to try and clear these conditions, and
high channel utilization may be experienced.

Recommendations:

To avoid the high channel utilization/performance concern, IBM recommends the fol-
lowing:
• First, ensure the RESET activation profile has the I/O Interface Reset option
enabled. Under normal shutdown and some disabled wait conditions, for z/VM
and z/OS, this will not only release any reserves but also prevent any status pending
conditions from occurring.
• Second, certain disable wait situations will not reset the I/O interface even when the
I/O reset option is enabled. These situations will require a manual SYSRST or
appropriate operator response.
• Clients should always do a SYSRST after shutting down
the operating system

Therefore, being aware of the state of the images, the state of the System Control Pro-
gram, and using zSeries best operational practices allows early corrective actions to be
taken thereby lessening any potential adverse effect.

2 of 11 Managing Logical Partition States

 Recommended Best Practices

4.0 Recommended Best Practices

The list of best practices, outlined below, cover many well known situations. These
practices, when followed, may prevent adverse impact to the system commonly seen as
performance degradation.

• In the RESET PROFILE, on the “Options” tab set to "on" the interface system reset
option as illustrated in the following figure:

Note:

Previous and existing versions of the PR/SM Planning Guide indicate that the
automatic I/O Interface option should not be used. The grounds for those recom-
mendations are no longer valid so the PR/SM Planning Guide will be updated to
indicate that the setting of the automatic I/O interface reset should always be
enabled.

Managing Logical Partition States 3 of 11

 Logical Partition States

• Use the activate task to initialize the CPC and not the POR task. The POR task will
not perform the interface system reset even if enabled.
• Never stop CPs for an image if there are any production or development images run-
ning on the same zSeries processor.
• Avoid removing (unplugging) fiber channel cables without first configuring off the
channel physically from all logical partitions in the enterprise that may have access.
The operating system commands should always be used. Only as a last resort should
the channels be configured offline via the Support Element task.
• Automate the use of the z/OS command: 'D M' at IPL time. This will provide a
snapshot of the system in the logs in the event problems occur and problem determi-
nation is required.
• Use the z/OS command: 'D M=CONFIG(xx)' at IPL time and once per shift in order
to check and verify any configuration deviations
• Respond to all Hardware Management Console (HMC) hardware message present
conditions
• Respond to all Hardware Management Console operating system message condi-
tions
• Keep the exceptions view area of the Hardware Management Console "green" by
responding to any "red" exception condition. (not responding to recommended
actions could lead to a adverse system impact)
• Ensure there are sufficient operational Hardware Management Consoles in the oper-
ations area to maximize operator awareness of an exception condition

5.0 Logical Partition States

The following sections and tables show the various LPAR image states and the affect
those states have on the channel I/O subsystem as well as the recommended action that
should be taken.

It should be noted that before any discussion concerning partition states must assume
that the underlying hardware platform is functioning correctly. The CPC state must be
operating with no CPC exceptions before logical partitions can be activated. The CPC
Details Panel can be checked to determine the state of the CPC or a review of hardware
messages can be done.

The following should also be noted:

• Storage, CPs, and other processors (ICFs,IFLs, and zAAPs) are assigned once a log-
ical partition is activated. It is possible and probable that an IPL or load of the parti-
tion will occur sometime after the image is activated.

4 of 11 Managing Logical Partition States

 Logical Partition States

• Subchannels are not valid until a logical partition image is activated.

• Logical path initialization is not attempted until a logical partition image is acti-
vated.
• I/O logical paths are established between logical partition images and the defined
control unit images once an image is activated.

5.1 Logical Partition States where a problem has been encountered.

Table 1: Disabled Wait State

Logical Logical Logical Logical I/O Subsystem Preferred

Partition Partition Partition Partition state for image Action
State Resource State Subchannel Logical Path
State State

Non-restartable Storage, CPs, Disabled Established Not in a path Determine the

disabled wait IFLs, ICFs, and group to this meaning of the
state. The HMC zAAPs are logical partition wait state code
shows the wait assigned as but the and correct. No
state message defined in the I/O may present other immediate
with the IO image profile. I/O status. Status additional system
Interface reset (Note 1) will be accepted action is
message. and discarded. required.

Non restartable Storage, CPS, Enabled Established Can present I/O Determine the
disabled wait IFLs, ICFs, status. However, meaning of the
state. The HMC zAAPs are status cannot be state code and
shows the wait assigned per the presented to the correct.
state message image profile. SCP (z/OS).
but with no I/O (Note 1) (Note 2) System reset the
interface reset logical partition.
message. (Note 3)

Restartable Storage, CPs, Enabled Established Can present I/O Determine the
disabled wait ICFs, IFLs, and Status. However, meaning of the
state. The HMC zAAPs are status cannot be wait state code
shows the wait assigned per the presented to the and perform the
state message image profile. SCP (z/OS) recommended
but no I/O (Note 1) (Note 2) actions.
interface reset
message

Notes:
1. Resources can only be unassigned by SCP commands after the system has com-
pleted the IPL process or the partition is deactivated.

Managing Logical Partition States 5 of 11

 Logical Partition States

2. Multiple status messages from the same device will be stacked and can cause overall
system performance problems on the same processor. IBM Poughkeepsie channel
development is trying to address the impact of this condition.
3. It is strongly recommended that the interface reset option in the RESET profile be
enabled.

Table 2: Poor Operating Conditions

Logical Logical Logical Logical I/O Subsystem Preferred
Partition State Partition Partition Partition state for image Action
Resource State Subchannel Logical Path
State State

Logical Partition Storage, CPs, Enabled Established Can present I/O The stopping of
CPs placed in a and other Status. However, logical CPs
processors status cannot be should only be
stopped state
(ICFs, IFLs, and presented to the performed for
zAAPs) are SCP (z/OS) development and
assigned per the (Note 2) test systems
image profile. under strict
(Note 1) control of the
overall system
owner. It should
never be done
when there are
production level
partitions active
on the same
processor.

Notes:
1. Once the partition image has been activated, resources can only be unassigned by z/
OS commands (when the system is operational) or by deactivating the image.
2. Multiple status conditions from the same device will be stacked and could adversely
affect the channel utilization.

6 of 11 Managing Logical Partition States

 Logical Partition States

Table 3: IPL Failures

Logical Logical Logical Logical I/O Subsystem Preferred

Partition State Partition Partition Partition state for image Action
Resource State Subchannel Logical Path
State State

Image activated. Storage, CPs, Disabled (except Established Not in a path Check correct
IPL not complete ICFs, IFLs, and IPL service) group to this load device
due to CC3 zAAPs are image but may number.
condition (i.e. IPL assigned per the present I/O Check path or
device or paths image profile status to this paths to the load
are not (Note 1) image. device (Problem
operational) (Note 2) determination
link analysis).
Check the control
unit interface and
power state.

Image activated, Storage, CPs, Disabled (except Established Not in a path System Reset
IPL not complete ICFs, IFLs, and the IPL device group to this
due to an I/O zAAPs are subchannel) image but may
error on the IPL assigned per the present I/O
device, due to a image profile. status to this
unit check, or (Note 1) image.
Interface Control (Note 2)
Check for the IPL
device.

Managing Logical Partition States 7 of 11

 Logical Partition States

Table 3: IPL Failures

Logical Logical Logical Logical I/O Subsystem Preferred

Partition State Partition Partition Partition state for image Action
Resource State Subchannel Logical Path
State State

Image activated, Storage, CPs, Some or all Established Not known 1. The state of
IPL complete, ICFs, IFLs, and channels the logical
SCP state not zAAPs are enabled partition image
known (i.e. No assigned per the icon should
SCP messages image profile. indicate there are
displayed) (Note 1) no messages
waiting.
2. Check that the
image is not in a
disabled wait
state.
3. Check that for
each possible
console there are
no messages
being displayed.
4. For z/OS,
perform a
standalone
dump.
5. Perform a
System Reset
after SAD
completes.

Notes:
1. Resources can only be unassigned by SCP commands after the system has com-
pleted the IPL process. Resources can be released by a deactivation of the partition.
2. Status will be accepted and discarded. An I/O error state (for the IPL device) may be
held in the control unit and “lock out” other systems.

5.2 Logical Partition States-Normal

The following tables show various normally encountered activation states and running
states.

8 of 11 Managing Logical Partition States

 Logical Partition States

Table 4: Activation States

Logical Logical Logical Logical I/O Subsystem Preferred

Partition State Partition Partition Partition path state for image Action
Resource State Subchannel state
State

CPC operating Storage, CPs, Disabled. Not established. N/A Verify that the
with no ICFs, IFLs, and processor is in
exceptions, zAAPs are not the required
image not allocated to the operational state.
activated image. (Note 1)
Activate the
logical partitions
per the client’s
requirements.

CPC activated None Assigned Disabled. Not established. N/A Determine from
with no the activation
exceptions, failure message
image failed to the cause of the
image activation
activate due to
failure. Correct
possible
as needed.
resource (Note 2)
shortage

CPC Activated Storage, CPs, Disabled Established Not in a path As per the client’s
with no ICFs, IFLs, and group to this operational
exceptions, zAAPs are image but control requirements
partition is allocated to the units may
image per the present I/O
activated and in
image profile. status.
the initial reset
(Note 3) (Note 4)
state. Image
logical CPs are
in the manual
state

Notes:
1. If needed, check the CPC Details panel on the Hardware Management Console or
the CPC hardware messages task.
2. This type of situation may be encountered in a case where HSA has consumed more
resource than planned and there is insufficient storage to activate the image. It may
also be encountered if there are insufficient processors available to allocate to the
image.
3. Resources can only be unassigned by SCP commands after the system has com-
pleted the IPL process. Resources can be released by a deactivation of the partition.

Managing Logical Partition States 9 of 11

 Logical Partition States

4. I/O status will be accepted and discarded by the image channel. In an overly defined
configuration, other systems may get a CC3 while this image is activated. Overly
defined in this context has to do with the number of logical paths defined. Logical
paths to a control unit are initially established at IMAGE activation. When an image
is activated, a logical path to that control unit is established and is kept even if the
image is in a SYSRESET condition. If there are a number of these images that are in
this SYSRST condition, this prevents other images from getting access to this
resource because all of the logical paths have already been established at the control
unit end.

Table 5: Normal Running States

Logical Logical Logical Logical I/O Subsystem Preferred

Partition State Partition Partition Partition state for image Action
Resource State Subchannel Logical Path
State State

Partition IPL-ed Storage, CPs, Enabled Established Devices are in a The SCP (z/OS
and in a normal ICFs, IFLs, and path group to this or z/VM) should
SCP (z/OS or zAAPs are image and can be responding to
z/VM) running allocated to the present I/O operator
image per the status. commands.
state
image profile. (Note 2) Perform normal
(Note 1) operator
procedures.
(Note 3)

Partition is Storage, CPs, Disabled Established Not in a path The image

System Reset ICFs, IFLs, and group to this details panel will
state zAAPs are image but the I/O show the CP in a
allocated to the may present I/O not operating
image per the status. status. Operate
image profile. (Note 4) as per the client’s
(Note 1) requirements.
Deactivate the
image if not
required.
(Note5)

Notes:
1. Channels are assigned as per the active I/O configuration file (IOCDS). Storage,
CPs, and other processors (ICFs, IFLs, zAAPs) and channels may be unassigned by
the z/OS CONFIG OFF commands, e.g. “CF CHP(cc), OFF”, where cc=CHPID
number. For coupling facility images, use the CFCC operator commands.
2. This status can “I/O Interrupt” the SCP (z/OS or z/VM) and will be handled by the
SCP.
3. For example, monitor the Hardware Management Console for exception conditions
on the CPC or image icons.

10 of 11 Managing Logical Partition States

 Summary

4. Status will be accepted by the channel and discarded. In an overly defined configura-
tion, other systems may get a CC3 (non-operational condition).
5. Deactivating an image will affect the weighting and capping values for the overall
processor.

6.0 Summary
Further information on this subject and other zSeries related topics can be found at the
following web sites:
• www.ibm.com/servers/resourcelink
• www.ibm.com/support/techdocs/atsmastr.nsf/Web/Techdocs

Managing Logical Partition States 11 of 11