TS7740 Training 2017-05 - Ver 1.0 PDF

Hardware related training for TS7740
 HW overview
 How to set up hardware of TS7740
 How to recover TS7740 from scratch
 How to analyze and repair TS7740
 Known constraints and limitations
 Content of USB Key & Reference Materials
1
Time Table
The offered training is based on 3 days
 Day 1 = theory, slides
 Day 2 = practice, installation
 Day 3 = practice, installation & error handling
 (Day 4 = open questions)
2
TS7740
 HW overview
3
Advanced
TS7740 Technical Technical
Skills Skills
TS7740 Virtualization Engine Components

 TS7740 Virtualization Engine (3957 Model V07)
– Power7 Atlas HV32 Server
– One 3Ghz 8 processor core card, 16GB Memory
– Runs the V and H nodes (Gnode)
– Integrated Enterprise Library Controller
 TS7740 Cache Drawer (3956 Model CX7)

– 16 15K 600GB FC HDDs
– 7.04 TB usable capacity (after RAID 5 and spares)
– Three maximum drawers – Total 21.12 TB additional
 TS7740 Cache Controller (3956 Model CC7)

– Disk RAID array controller
– 16x 15K 600 GB FC HDDs
– 7.04 TB usable capacity (after RAID 5 and spares)
– Adding up to three CX7 drawers provides up to 28.17 TB total
capacity
 3952 Model F05 Frame

– Houses major components & support components The combination of the
– Dual Power Virtualization Engine components
is called a TS7700 Cluster
4
Advanced
Skills Skills
The Engines of the TS7740

 V07 = Power7 Atlas HV32 Server (As shipping today)
– One 3Ghz 8 processor core card, 16GB Memory for GA
– Running IBM AIX as host for the Firmware
 CC7 = DS4700 Storage Subsystem (As shipping today)

– 16x 600GB15K HDDs
 CX7 = EXP810 Storage Expansion(As shipping today)

– 16x 600GB15K HDDs
5
Advanced
Skills Skills
Mainframe Host TS7740 TS7700
Four FICON Paths TCP/IP

LAN/WA
N
TCP/IP
TS7700
Two 8 Gbs
Fibre Channel
Fibre Switch
Note: TS7740 by definition requires
Fibre Channel an IBM TS3500 Tape attached
3592 Tape Drive
(Without attached Tape you have TS7720)
6
Advanced
Skills Skills
TS1130/TS1140 Support (3592-E06/E07)

 All drives in the system must be of the same type
– 3592-J1A or 3592-E05 in J1A emulation mode
– 3592-E05 in native mode
– 3592-E06/EU6
– 3592-E07
 Media type/Capacities
Drive Type Media Capacity

3592-J1A No longer available JJ 60 GB
JJ 100 GB
3592-E05/TS1120 JA 500 GB
JB 700 GB
JJ 128 GB
3592-E06/TS1130 JA 640 GB
JB 1000 GB
JA Read Only Support in 2012 640 GB
3592-E07/TS1140
JB 1600 GB
Supported 9/9/2011
JC 4000 GB
7
TS7740
 How to set up hardware of TS7740
8
Advanced
Skills Skills
How to set up hardware of TS7740

1st Step - Prepare the system for installation
– Clean V0x disks (POWER7)
– Clean Cxx disks (DS4700)
– Check customer network ports
– Attach tape system (TS3500)
• Configure logical library
• Add drives
• Check control path
• Add tapes to logical library
– Connect notebook
• Serial port
• Network for storage
9
Advanced
Skills Skills
1. prepare the system

 1.1 Clean V07 (POWER7) disks
For the new installation of a TS7740 it is necessary to use empty HDDs.
Here are 2 possible ways to clean the disks.
1. Option use AIX DIAG CD (slow)

1. Boot a AIX DIAG and use the “format disk” routine in the advanced service menus
2. You can format the HDDs one by one
3. Advantage: no need of additional hardware, the format process checks the disks
4. Disadvantage: time requirement, each disk needs 15-20 min to format (without certify)
2. Option use a external LINUX PC (very quick)
1. You need an external PC/Workstation/Server (x86) with SAS controller(s) and single SAS cable
2. Remove the disks from the V07 and connect the disks to the external Workstation
3. Boot the workstation with a Linux live cd (like Ubuntu Server Install CD)
4. Use lsscsi to check the available HDDs and identify the name (like /dev/sdb)
5. For each V07 SAS Disk run as root wipefs –a /dev/sd.. (like wipefs –a /dev/sdb)
6. Advantage: very fast, about 10min for all disks
7. Disadvantage: you need additional hardware
10
Advanced
Skills Skills

 1.2 clean CC7 (DS4700) disks (1 of 5)
- Connect Laptop and CC7 via direct Ethernet path

(no switch required)
- Use port CC7 CB P1 (red mark)
- Set Laptop Network to e.g.192.168.128.1
- Exceptional 192.168.128.102 / 192.168.128.101
11
Advanced
Skills Skills

 1.2 clean Cxx (DS4700) disks (2 of 5)
- Start IBM Storage Manager (Laptop)

- Search for the CC7
(DS4700 Storage Subsystem)
12
Advanced
Skills Skills

- Select logical drives

- Select first raid group
- Choose delete
13
Advanced
Skills Skills

- Select arrarys
- Confirm delete
- Make sure all arrays are gone
14
Advanced
Skills Skills

- Select Physical
- Select all (4) hot spares
- Select Hot Spare Coverage
- Select Manually unassign
- Confirm
- Make sure all spares are gone
15
Advanced
Skills Skills

 1.3 Check customer network ports
• Connect both ethernet ports

• Both ports need a link signal
16
Advanced
Skills Skills

 1.4 attach a tape system (1 of 2)
• Create a logical library
• Assign 2 or more 3592 drives
• Assign some cartridges
• Connect both ports of each

drive to different FC switch
• Port 0 -> primary switch
• Port 1 -> alternate switch
• Start populating from port 1
of each switch
17
Advanced Technical Skills (ATS) North America

 1.4 attach a tape system (2 of 2)
• Connect V07 ports to FC
switch
• 251 PRY I/O C4 Port 0

-> PRY-SW Port 19 (primary)
• 250 ALT I/O C4 Port 0

-> ALT- SW Port 19 (alternate)
• Use EXACTLY above

indicated ports
18 © 2011 IBM Corporation

Advanced
Skills Skills

 1.5 connect the service laptop (serial port)
• V07 (POWER7) use serial port 1

• Speed 19200/8/N/1
• Recommended emulation ibm3151
At this point all hardware preparations

for install are completed
19
TS7740
 How to recover TS7740 from scratch
based on MC = 8.21.0.165
20
Advanced
Skills Skills
Overview - TS7740 Recover from scratch

2. Install operating system 3. Activate code 4. Finishing the installation
• With backup DVD-RAM • Login to system
– Boot from DVD
• Login to the system • activate necessary feature
– Start install • Start activate • enter necessary feature keys
– Change serial number • Access the Backup-DVD- • complete customer network
RAM configuration
• Boot to AIX root shell • run check out for failure and
• Check configuration
• Edit file for serial number change warnings
• Without Backup-DVD-RAM
• reboot • find failures and warnings
• Login to the system and recognize the meaning
• Start activate • open an web connection to
• Skip Backup-DVD-RAM the system
• Enter the configuration • create backup
• update microcode
21
Advanced
Skills Skills
2. install operating system (1 of 6)

 Open a terminal emulation
 (e.g. NetTerm)
 Power on the V07 (POWER7)
 Insert the 1st. MC DVD
 At boot menu press „5“ for
„Default Boot List“
 Type 2 to continue
 Enter „admin“ as password for
user admin.
22
Advanced
Skills Skills

 Confirm the preselection (1) on
the next screens
 When prompted change DVD
 After the V07 has completed the
DVD-copy, the V07 automatically
restarts
 [OPTION 1] backup DVD is NOT
available - go to chapter 2.2
 [OPTION 2] backup DVD is
available - go to chapter 3
23
Advanced
Skills Skills

 2.2 NO backup DVD - change serial number
For installations without a backup DVD it is necessary to change the V07 serial number
This procedure MUST be performed before activating
• At V07 reboot, insert AIX (5/6/7)

Installation DVD
• At boot menu press „5“ for
„Default Boot List“
• Type 2 to continue
• Enter „admin“ as password for
user admin
24
Advanced
Skills Skills

• Key: “1”, this is the system console.
• Key: “1” for English.
• Key: “3” for “Start Maintenance Mode for
System Recovery”
• Key: “1” for “Access a Root Volume Group”
• Key: “0” to Continue
• Key: “1” to choose the hdisk0
• Key: “1” to “Access this Volume Group and
start a shell”
• After some minutes you get a prompt “#”
25
Advanced
Skills Skills

• Define the terminal emulation e.g. ibm3151

• Type in “export TERM=ibm3151” and press enter
• Change working directory
• Type “cd /usr/vtd/scripts/” and press enter
• With an editor of your choice (e.g. VI) open the file “vtd_vpdcfg”
• Type in “vi vtd_vpdcfg” and press enter
• Type in “/ V06_serial” and press enter
• Find the section “function query_V06_serial”
• Set the cursor to the line starting with “V06_serial=$(lscfg….”
• Press “o” to insert a new line and enter the edit mode
• Enter the following “V06_serial=H5523”
• Press ESC to leave the edit mode
• Enter “:wq” to write the changes and exit
• Reboot the system “shutdown –r now”
26
TS7740
3. activate micro code
27
Advanced
Skills Skills

3.1 start activation
• After reboot login as user Service or Support
• Key: “3” for IBM Console (if you use NetTerm).
• Key: “a” for activate microcode
28
Advanced
Skills Skills

3.1 start activation
• After some time the following will appear:
MSG : Please insert the 'Configuration Backup' DVD-RAM

into the bottom DVD drive (/dev/cd1):
+--------------------------------------------+
| DVD-ROM o #### +---+
+----------------------------------------+
+--------------------------------------------+ [OPTION 1] backup
==> | DVD Multi Recorder o #### +---+ DVD is available - go to
+----------------------------------------+ chapter 3.2
Press 'c' and [Enter] to continue
or 'e' and [Enter] to exit: [OPTION 2] backup
DVD is NOT available -
go to chapter 3.3
29
Advanced
Skills Skills

3.2 start activation [with Backup-DVD]
• Insert the backup-DVD into the bottom drive and enter c to continue
• The V07 will load the necessary information from the DVD
• After loading, the modifiable information data will be displayed
• You now have the option to customize this data
• After changing the data as needed select “c” to continue
30
Advanced
Skills Skills

3.3 start activation [without Backup-DVD]
• If you do not have a backup DVD Key: select “e” and [Enter] to exit
• Ignore the warning and select Key: “c” and [Enter] to continue without vital product data
• After some time the V07 will ask to complete the missing VPD information as follows
• - clusterindex 0 (identify the VTS in cluster, unique in cluster)
• - frame serial xxxxx (your F05 rack frame serial number without 78)
• - composite_lib_sequence LIB01 (must be different to the other TS7740)
• - distributed_lib_sequence LIB11 (must be different to the other TS7740)
• - libraryattach 3584
• - fcp_switch_type1 imbedded
• - E05 drive emulation J1A no
• - type of cache 2 (for DS4000)
• - cache_serial 78xxxxx (CC7/DS4000 S/N)
• - additional cache controllers 0
• - tssc ip 172.31.1.160 (insert your address)
• -type of internal network 2 (for switch)
• Then check and confirm your choices.
31
Advanced
Skills Skills

3.4 check and finish activation
• The microcode activation runs about 2-3 hours
• If no important failure appear the service menus now become available
• Please reboot the V07 system an logon as Support
32
TS7740
4. Finishing the installation
33
Advanced
Skills Skills
4. finishing the installation

 4.1 install features codes
• select Manufacturing Menus
• select Install Feature Codes
• select the features you need

• select at least FC 9000, FC5240 respectively FC5241
• Press F4 to list possibilities
• Press Enter to commit and F3 to exit the menu
34
Advanced
Skills Skills

 4.2 install feature keys
• select Manufacturing Menus
• select Install Feature Key
• Enter the 32 character Feature Key comes with the system
• Install at least 1x FC5267 1TB CACHE ENA. and 1xFC5268 100MB/SEC INCR.
• Press Enter to commit and F3 to exit the menu
• You can add feature keys later trough the web GUI
• Please reboot the system
• E.g. for S/N 78-H5523

• 5267 - 1TB Cache Enablement - INS 1 - d85927c06b005a53c6402a0d31221121
• 5268 - 100 MB/SEC Increment - INS 1 - d85927c06b005a537baf4e24d52c42b6
• 4015 - GRID Enablement - INS 1 - d85927c06b005a5330af0ba13ef69365
35
Advanced
Skills Skills

 4.3 configure the customer network
• select Support Center Menus
• select IBM TS7700 Virtualization Engine Maintenance
• select Subsystem Configuration Menus
• select Customer Network Configuration Menus
• select Set Full Customer Network Configuration
Fill out the form - example:
Is this the FIRST time that Customer Network [Yes]
* Enter the Cluster's Virtual Customer IP: [xxx.xxx.xxx.xxx] (customer choice)
NOTE: This IP is the one that the customer will use to connect to the cluster.
* Enter the Primary Customer IP: [xxx.xxx.xxx.xxx] (customer choice)
* Enter the Alternate Customer IP: [xxx.xxx.xxx.xxx] (customer choice)
Enter the Customer's Gateway: [xxx.xxx.xxx.xxx] (customer choice)
* Enter the Customer's Subnet Mask: [xxx.xxx.xxx.xxx] (customer choice)
* Customer Network Speed/Duplex Setting: Auto Negotiate
Enter the Customer's 3584 IP: [xxx.xxx.xxx.xxx] (customer choice)
Enter the Service Console IP Address: [xxx.xxx.xxx.xxx] (TSSC / TS3000)
Enter the Service Console Hostname (TSSC): [tssnet1]
Enter the TS7700 IP Address to be : [xxx.xxx.xxx.xxx]
• Press enter to commit
36
Advanced
Skills Skills

 4.4 check the machine status
• after reboot it takes about 20-30min to start all components, be patient
• select Support Center Menus
• select IBM TS7700 Virtualization Engine Maintenance
• select System Checkout Menus
• Select Display Current System Status
• The routine starts and shows the results after a few moments
• On the top screen you should see the VNODE and HNODE status
• Online = the system is ready for access via WEB GUI (there may still be tasks to resolve)
• In transition to online = the system has not yet started all processes, wait a few minutes and repeat
• Offline = There are major problems that prevent going Online
• Scroll down to see Summary of NON-OK Statuses
• Use / to find the corresponding section an find more information about the problem
37
Advanced
Skills Skills

 4.4 open a web connection to the system
• After the system is Online, open a web browser on a client computer
• Go to the following address: xxx.xxx.xxx.xxx/y/Console
• Xxx.xxx.xxx.xxx = is the Cluster's Virtual Customer IP defined in chapter 4.3
• y = clusterindex - which was defined during activation
• E.g. 192.168.0.1/0/Console
• Connect to the web gui and get a login screen
• Login as admin with password admin
• The TS7740 hardware installation is ready

at this moment
• The next step is configuration and fine tuning

(these tasks are NOT part of this training)
38
TS7740
5. Final steps
39
Advanced
Skills Skills
5. Final steps
 5.1 backup
Once you have an installation without errors, make a backup and store the DVD in a secure place.
Create a 2nd backup DVD to remain in the machine for automatic backup.
40
Advanced
Skills Skills
5. Final steps
 5.1 update micro code
• Go to the Microcode Maintenance Menus –> TS7700 Microcode Update Menus
• 1st. step – copy the newest available micro code to the system
• 2nd. step - activate the new code
• 3rd. step - after successful update, remember to repeat backup
• In a cluster solution all systems *MUST* be on the same micro code level
41
TS7740
 How to analyze and repair TS7740
42
Advanced
Skills Skills
TS7740 Troubleshooting
TS7740 trouble shooting
requires AIX skills !
For trouble shooting

you have the following
tools
 V07 Operator panel display
for hardware issues
 AIX SMIT error log
via user „Support“
 AIX Command line (CLI)
via user “Enhanced” to
allow full access
43
Advanced
Skills Skills
Operator panel display of the V07
The V07 operator panel shows

system health status via codes.
The codes help identifying

hardware defects of the V07
i.e. processor damage (code 8410)

i.e. HBA damage (code 22000001)
i.e. Power Supply damage (code 40100100)
The following link leads to the

documentation of the error codes:
https://www.ibm.com/support/knowledgecenter/POWER5/ipha6_p5/ipha6.pdf
Very good! No error code on the Operator panel display

44
Advanced
Skills Skills
How to view the SMIT logfile

Note that the path to find log files
2. and/or error logs is complicated.
1.
The following slides guide you.
1. Support Center Menus
3.
2. IBM TS7700 Virtualization
Engine
3. SIM, Error Log, Diagnostics
4. Subsystem Error Log Menus
4.
5.
5. Generate an Error Report
45
Advanced
Skills Skills
Switch from sumary to detailed error reporting:
46
Advanced
Skills Skills
- At this moment you are in the

(detailed) Error log
- Use / to search & find
- Search for „LABEL“ for jumping
to the next error
47
Advanced
Skills Skills
Hit „n“ for „Find Next“ [LABEL]… until …
Gotcha!
We found a log
entry with an error
Here: missing FC 5241
48
Advanced
Skills Skills
Typical error messages

Example 1
Storage not wiped or defective hard disc
Comment: often the installation fails, due to
remaining data fragments on HDDs !
(Note: hanging actfinis script for activation)
Example 2
Power supply defective
49
Advanced
Skills Skills
Typical error messages of known typical errors

a.) Unable to find the Primary and Alternate switches (or routers).
Verify that:
- All network cables are properly connected
- The switches (or routers) are both powered on
b.) Thu 02/04/16 15:35:20 - ERROR: No tape devices found.

c.) ERROR: 3584 control path configuration problem.
d.) Thu 02/04/16 15:35:53 - MSG : No Primary grid IP address set, not creating rout.
e.) Thu 02/04/16 15:35:53 - MSG : No alternate grid IP address set, not creating ro.
f.) Thu 02/04/16 15:36:21 - Detected one or more hardware errors(s).
g.) Thu 02/04/16 15:47:38 - ERROR : 'error' deferred activation events were created
act_finis. Start_Node will be skipped
h.) Thu 02/04/16 15:48:00 - MSG : skipping online because one or more deferred ERRORS
50
Advanced
Skills Skills
Hacker’s corner
root:piff1e
Passwords known to RuP: Support:ibmsup4u
hdbuser:db2user
hdbtemp:db2user
ermm:db2user
service:ibm2serv
IBM2_serv (CC7 Storage Manager)

service:service (TS3000)
admin:admin (Service Processor)
51
Advanced
Skills Skills
Become the user “Enhanced”

1. Boot with AIX-DIAG-CD Why would I need the user
2. Mount the root filesystem “Enhanced” at all?
3. Remove the password of the User "Enhanced": You need the user „Enhanced“
to access CLI in the cases the
4. set -o emacs GUI either fails or cannot
5. export TERM=vt100 (or ibm3151) resolve your issue
6. cd /etc/security Note: dangerous - only use

user „Enhanced“ as last resort
7. cp -p passwd passwd.orig
8. sed 's/F51006yRWoIxo//g' < passwd.orig > passwd You better have deep AIX
knowledge
9. Reboot AIX and log in as Enhanced – bingo 
52
TS7740
Known constraints and limitations
53
Advanced
Skills Skills
7. known problems and limitations

 7.1 Microcode
• New install with code 8.21.0.165 will not come online

• reason: the error message reports missing feature 5241
• If you install this feature as described in section 4.1, the function runs
error free but the system does not recognize it as installed
• Solution install code 8.20.0.123 and upgrade to 8.21.0.165 or use backup
DVD
54
Advanced
Skills Skills

 7.2 TS3000
• A TS3000 console is required to operate the TS7740
• No dedicated TS3000 is required (you can share TS3000)
• You can use already existing consoles
• In this case, you must adjust the IP settings
55
Advanced
Skills Skills

 7.3 GRID
• The GRID function has the known restriction that (for non-IBM parties) it is not
possible to return to stand alone system
• Moving from GRID requires initial setup
• The GRID function has its risks, we advise to not use this
56
Advanced
Skills Skills
Reference Materials
 Techdocs
– http://www-03.ibm.com/support/techdocs/atsmastr.nsf/Web/TechDocs
– Search for TS7700 whitepapers and presentations
 Redbooks
– http://www.redbooks.ibm.com/
– Search on TS7700
 TS7700 Infocenter
– http://publib.boulder.ibm.com/infocenter/ts7700/cust/index.jsp
– Managing -> Management Interface
57
Advanced
Skills Skills
8. Content of USB Key

 TS7740 Training (PPT file)
 MC TS7740 - 8.20.0.123 (2x DVD, base)
 MC TS7740 - 8.21.0.165 (2x DVD, upgrade
 MC TS3000
 Netterm (for 3151 emulation)
58
Advanced
Skills Skills
Questions?
59

TS7740 Training 2017-05 - Ver 1.0 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

TS7740 Training 2017-05 - Ver 1.0 PDF

Uploaded by

Copyright:

Available Formats

Hardware related training for TS7740

TS7740 Virtualization Engine Components

 TS7740 Cache Drawer (3956 Model CX7)

 TS7740 Cache Controller (3956 Model CC7)

 3952 Model F05 Frame

The Engines of the TS7740

 CC7 = DS4700 Storage Subsystem (As shipping today)

 CX7 = EXP810 Storage Expansion(As shipping today)

Mainframe Host TS7740 TS7700

Four FICON Paths TCP/IP

TS1130/TS1140 Support (3592-E06/E07)

Drive Type Media Capacity

How to set up hardware of TS7740

1. prepare the system

1. Option use AIX DIAG CD (slow)

1. prepare the system

- Connect Laptop and CC7 via direct Ethernet path

1. prepare the system

- Start IBM Storage Manager (Laptop)

1. prepare the system

- Select logical drives

1. prepare the system

- Make sure all arrays are gone

1. prepare the system

- Make sure all spares are gone

1. prepare the system

• Connect both ethernet ports

1. prepare the system

• Connect both ports of each

1. prepare the system

• 251 PRY I/O C4 Port 0

• 250 ALT I/O C4 Port 0

• Use EXACTLY above

18 © 2011 IBM Corporation

1. prepare the system

• V07 (POWER7) use serial port 1

At this point all hardware preparations

Overview - TS7740 Recover from scratch

2. install operating system (1 of 6)

2. install operating system (2 of 6)

2. install operating system (3 of 6)

• At V07 reboot, insert AIX (5/6/7)

2. install operating system (4 of 6)

2. install operating system (5 of 6)

• Define the terminal emulation e.g. ibm3151

3. activate micro code

3. activate micro code

MSG : Please insert the 'Configuration Backup' DVD-RAM

3. activate micro code

3. activate micro code

3. activate micro code

• If no important failure appear the service menus now become available

• Please reboot the V07 system an logon as Support

4. finishing the installation

• select the features you need

4. finishing the installation

• E.g. for S/N 78-H5523

4. finishing the installation

4. finishing the installation

4. finishing the installation

• The TS7740 hardware installation is ready

• The next step is configuration and fine tuning

For trouble shooting