You are on page 1of 186

Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
no Machine
Exadata Database
a
s
) ha eฺ Workshop
Administration
ฺ c om Guid
e chActivity
e n t
t
fo Stud Guide – Volume I
i n
e zy this
o h i@ use
s b irs e to
i ( ja ens
o h lic
b i rS
Ja s

D73668GC30
Edition 3.0
May 2014
D86662
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Disclaimer

This document contains proprietary information and is protected by copyright and other intellectual property laws. You may copy and
print this document solely for your own use in an Oracle training course. The document may not be modified or altered in any way.
Except where your use constitutes "fair use" under copyright law, you may not use, share, download, upload, copy, print, display,
perform, reproduce, publish, license, post, transmit, or distribute this document in whole or in part without the express authorization
of Oracle.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

The information contained in this document is subject to change without notice. If you find any problems in the document, please
report them in writing to: Oracle University, 500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not
warranted to be error-free.

Restricted Rights Notice

If this documentation is delivered to the United States Government or anyone using the documentation on behalf of the United
States Government, the following notice is applicable:

U.S. GOVERNMENT RIGHTS


The U.S. Government’s rights to use, modify, reproduce, release, perform, display, or disclose these training materials are restricted
by the terms of the applicable Oracle license agreement and/or the applicable U.S. Government contract. bl e
fe r a
Trademark Notice
an s
n - t r
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective
owners.
a no
Author
) h as ฺ
Peter Fusek
c o m uide
Technical Contributors and Reviewers e c hฺ ent G
Akshay Shah, Alex Tsukerman, Amit Ganesh,
i n fotAndrew
S t d Aslam Edah-Tally, Barb Lundhild,
uBabb,
y
z thisBranislav Valny, Bruce Kyro, Caroline Johnston,
Bharat Baddepudi, Bill Hodak, Boris Erlikhman,
e
Christian Craft, Dan Norris, Darryl
h i @ Utzig,
Presley,
s e Dave Winter, David Hernandez Mendoza, David
Hitchcock, Deba Chatterjee,
i r s o to u Ed Gilowski, Eric Siglin, Georg Schmidt, Harald van
Douglas

a s b seHe, James Womack, Jean-Francois Verrier, Jia Shi,Jignesh


Breederode, Ira Singer, James
Patel, Jim Hall,
i jJim Spiller,
(Umamageswaran,enJim Viscusi, Joel Goodman, Juan Loaiza, Kam Shergill,Kevin
Jernigan, h l i c
o To, Louis Nagode, Mahesh Subramaniam,
Kodi Krishnanjani Chitta, Lachlan Williams, Larry Justice,
S
ir Scardina, Mark Van de Wiel, Marshall Presser, Martin
Lawrence Maria Billings, Maria Colgan, Mark Fuller,
a s b
Mark Jensen, Michael Cebulla, Michael
J Nowak, Naoki Kato, Nilesh Choudhury, Ravindra Dani, Raymond Dutcher, Richard Exley,
Robert Carlin, Robert Pastijn, Roger Hansen, Sabyasachi Banerjee, Sean Kim, Selcuk Aya,
Scott Gossett, Sriram Palapudi, Steven Lemme, Sue Lee, Sugam Pandey, Sumeet Lahorani,
Sundararaman Sridharan, Tim Shelter, Umesh Panchaksharaiah, Uwe Hesse, Varun Malhotra,
Vern Wagman, Vijay Sridharan, Vikram Kapoor, Vimala Jacob

This book was published using: oracletutor


Table of Contents
Practices for Lesson 1: Introduction ..............................................................................................................1-1
Practices for Lesson 1....................................................................................................................................1-2
Practice 1-1: Lab Environment Introduction ...................................................................................................1-3
Practices for Lesson 2: Exadata Database Machine: Overview ...................................................................2-1
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Practices for Lesson 2....................................................................................................................................2-2


Practices for Lesson 3: Exadata Database Machine Architecture ...............................................................3-1
Practices for Lesson 3....................................................................................................................................3-2
Practice 3-1: Process Familiarization .............................................................................................................3-3
Practice 3-2: Exadata High Availability...........................................................................................................3-6
Practice 3-3: Storage Object Familiarization ..................................................................................................3-11
Practice 3-4: Exadata Smart Flash Cache Familiarization .............................................................................3-19
Practices for Lesson 4: Key Capabilities of Exadata Database Machine ....................................................4-1
ble
Practices for Lesson 4....................................................................................................................................4-2
fe r a
s
Practice 4-1: Smart Scan ...............................................................................................................................4-3
an
Practice 4-2: Exadata Hybrid Columnar Compression ...................................................................................4-7
n - t r
a no
Practice 4-3: Exadata Smart Flash Cache .....................................................................................................4-10
Practice 4-4: Storage Index ............................................................................................................................4-19

) has eฺ
Practices for Lesson 5: Exadata Database Machine Initial Configuration ..................................................5-1
o m uid
Practices for Lesson 5....................................................................................................................................5-2
c
Practice 5-1: Using the Oracle Exadata Deployment Assistant
c ฺ t G
h......................................................................5-3
n
Practices for Lesson 6: Exadata Storage Server Configuration
e de
fot Stu..................................................................6-1
y i n
e z t h is
Practices for Lesson 6....................................................................................................................................6-2

o
Practice 6-2: Storage Reconfigurationh i@...........................................................................................................6-6
Practice 6-1: Cell Configuration......................................................................................................................6-3
u s e
Practice 6-3: Consumingb rs Diskse bytoUsing ASM .......................................................................................6-25
iGrid
( s
ja Exadata ns Storage Security ......................................................................................6-34
Practice 6-4: Configuring
i c e
h Cell UserliAccounts....................................................................................................................6-49
Practiceo6-5:
ir S
Practice 6-6: Using the Distributed Command-Line Utility (dcli) .....................................................................6-51
s b
Ja Practices for Lesson 7: I/O Resource Management ......................................................................................7-1
Practices for Lesson 7....................................................................................................................................7-2
Practices for Lesson 8: Recommendations for Optimizing Database Performance ..................................8-1
Practices for Lesson 8....................................................................................................................................8-2
Practice 8-1: Configuring Write Back Flash Cache ........................................................................................8-3
Practice 8-2: Using Exadata Hybrid Columnar Compression .........................................................................8-9
Practice 8-3: Testing Index Elimination ..........................................................................................................8-22
Practices for Lesson 9: Using Smart Scan ....................................................................................................9-1
Practices for Lesson 9....................................................................................................................................9-2
Practice 9-1: Monitoring Exadata Smart Scan ...............................................................................................9-3
Practice 9-2: Monitoring Cell Wait Events for Parallel Query .........................................................................9-15
Practices for Lesson 10: Consolidation Options and Recommendations ..................................................10-1
Practices for Lesson 10..................................................................................................................................10-2
Practices for Lesson 11: Migrating Databases to Exadata ...........................................................................11-1
Practices for Lesson 11..................................................................................................................................11-2
Practice 11-1: Migrating to Databases Machine by Using Transportable Tablespaces..................................11-3

Copyright © 2014. Oracle and/or its affiliates. All rights reserved.

Exadata Database Machine Administration Workshop Table of Contents


i
Practices for Lesson 12: Bulk Data Loading by Using Oracle DBFS ..........................................................12-1
Practices for Lesson 12..................................................................................................................................12-2
Practice 12-1: Bulk Data Loading with Database Machine .............................................................................12-3
Practices for Lesson 13: Exadata Database Machine Platform Monitoring: Introduction .........................13-1
Practices for Lesson 13..................................................................................................................................13-2
Practice 13-1: Environment Reconfiguration ..................................................................................................13-3
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Practices for Lesson 14: Configuring Enterprise Manager Cloud Control 12c to Monitor Exadata
Database Machine ........................................................................................................................................14-1
Practices for Lesson 14..................................................................................................................................14-2
Practice 14-1: Configuring Enterprise Manager Cloud Control 12c to Monitor Exadata Database Machine ..14-3
Practice 14-2: Post-Discovery Configuration and Verification ........................................................................14-51
Practice 14-3: Environment Reconfiguration ..................................................................................................14-55
Practices for Lesson 15: Monitoring Exadata Storage Servers ...................................................................15-1
Practices for Lesson 15..................................................................................................................................15-2
Practice 15-1: Metrics, Alerts, and Active Requests.......................................................................................15-3 ble
Practice 15-2: Exadata Storage Server Monitoring with Enterprise Manager ................................................15-16 fe r a
an s
Practices for Lesson 16: Monitoring Exadata Database Machine Database Servers ................................16-1
n - t r
a no
Practices for Lesson 16..................................................................................................................................16-2
Practice 16-1: Exadata Database Monitoring with Enterprise Manager .........................................................16-3

has eฺ
Practices for Lesson 17: Monitoring the InfiniBand Network ......................................................................17-1
)
com uid
Practices for Lesson 17..................................................................................................................................17-2
ฺ G
Practice 17-1: Exadata InfiniBand Monitoring with Enterpriseh
e
Manager
c Machine e n t
.........................................................17-3
Practices for Lesson 18: Monitoring Other Exadatao
f t
Database
t u d Components ..............................18-1
y i n S
Practices for Lesson 18..................................................................................................................................18-2
e z h i s
h i @ se t .............................................................................19-1
Practices for Lesson 19: Other Useful Monitoring Tools

irso and u
Practices for Lesson 19..................................................................................................................................19-2
Practices for Lessons b
20: Backup e t o
Recovery ...........................................................................................20-1
( j a n s
hi Environment
Practices for Lesson
Practiceo20-1: iceReconfiguration ..................................................................................................20-3
20..................................................................................................................................20-2
l
b i r S 20-2: Backup Optimization ...............................................................................................................20-4
Practice
Jas Practice 20-3: Recovery Optimization ............................................................................................................20-12

Copyright © 2014. Oracle and/or its affiliates. All rights reserved.

Exadata Database Machine Administration Workshop Table of Contents


ii
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
s
haLesson
Practices )for
m i d eฺ 1:
Introduction
ฺ co t Gu
e h
c 1 den
o t
Chapter
y i nf S tu
ez thi s
h i @ s e
o
irs e to u
s b
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 1
Practices for Lesson 1
Practices Overview
In this practice, you will be introduced to the laboratory environment used to support all the
practices during this course.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 2
Practice 1-1: Lab Environment Introduction
Overview
In this practice, you will learn how to use the laboratory environment that supports all the
practices in this course.
The laboratory environment for this course is based on a Quarter Rack Database Machine. It
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

consists of several virtual machines configured to provide an Oracle Database server and three
Exadata Storage Servers. The second Oracle Database server, which is normally found on a
Quarter Rack Database Machine, is initially not started due to physical resource constraints.
To access the virtual machines, you will first establish a graphical session which is connected to
the VM server. Your instructor will provide specific details for each student's server. From there,
you will create terminal sessions as required and connect to the virtualized Database Machine
servers by using SSH as described in the tasks for this practice.

Tasks ble
fe r a
1. Establish a terminal session connected to the qr01db01 database server by using the
ans
grid operating system account.
n - t r
are prompted to acknowledge server authenticity. a no
Note that you may see additional messages relating to server identities. Answer yes if you

$ ssh grid@qr01db01 ) has eฺ


grid@qr01db01's password: <oracle>com
ฺ G uid
[grid@qr01db01 ~]$
t e ch dent
2. Execute the following command andin fo thatSalltuof the listed services are online in your
verify
e y
z should islook like the example below. Alert your
laboratory environment. Your output
instructor if you do not havei@the same t h
eservices online in your environment.
o h u s
b
[grid@qr01db01
s irs~]$ ecrsctl
to status resource -w "TARGET = ONLINE" -t
( ja eTARGET ns STATE
------------------------------------------------------------------------------
i c
h
NAME
li SERVER STATE_DETAILS
o------------------------------------------------------------------------------
S
a s bir Local Resources
J ------------------------------------------------------------------------------
ora.DATA_QR01.dg
ONLINE ONLINE qr01db01
ora.DBFS_DG.dg
ONLINE ONLINE qr01db01
ora.LISTENER.lsnr
ONLINE ONLINE qr01db01
ora.RECO_QR01.dg
ONLINE ONLINE qr01db01
ora.asm
ONLINE ONLINE qr01db01 Started
ora.net1.network
ONLINE ONLINE qr01db01
ora.ons
ONLINE ONLINE qr01db01
------------------------------------------------------------------------------
Cluster Resources

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 3
------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE qr01db01
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE qr01db01
ora.LISTENER_SCAN3.lsnr
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

1 ONLINE ONLINE qr01db01


ora.cvu
1 ONLINE ONLINE qr01db01
ora.dbm.db
1 ONLINE ONLINE qr01db01 Open
2 ONLINE OFFLINE
ora.oc4j
1 ONLINE ONLINE qr01db01
ora.qr01db01.vip
ble
1 ONLINE ONLINE qr01db01
fe r a
ora.qr01db02.vip
ans
1 ONLINE INTERMEDIATE qr01db01 FAILED OVER
n - t r
ora.scan1.vip
a no
has eฺ
1 ONLINE ONLINE qr01db01
ora.scan2.vip
)
1 ONLINE ONLINE qr01db01
ฺ c om Guid
ora.scan3.vip
t e ch dent
nfo Stu
1 ONLINE ONLINE qr01db01
[grid@qr01db01 ~]$
y i
3. Using SQL*Plus, connect to the e z thASM
clustered is environment as an ASM administrator.
h i@to uthesASM
Verify that you are connected
o e instance +ASM1 and exit your SQL*Plus session.
s b irs ~]$
[grid@qr01db01 e tosqlplus / as sysasm
i ( ja ens
S
h
oSQL*Plus: licRelease 11.2.0.3.0 Production...
a s bir
J SQL> select instance_name from v$instance;

INSTANCE_NAME
----------------
+ASM1

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release
11.2.0.3.0 - 64bit Production...
[oracle@qr01db01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 4
4. Establish a new shell as the oracle OS user.
[grid@qr01db01 ~]$ su - oracle
Password: <oracle>
[oracle@qr01db01 ~]$
5. Using SQL*Plus, connect to your database as the database administrator. Verify that you
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

are connected to the DBM database and exit your SQL*Plus session.
[oracle@qr01db01 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production...

SQL> select name from v$database;

ble
NAME
fe r a
---------
ans
DBM
n - t r
a no
has eฺEdition Release
SQL> exit
)
uid
Disconnected from Oracle Database 11g Enterprise
11.2.0.3.0 - 64bit Production... com

ch dent G
[oracle@qr01db01 ~]$
t e
i nfoof theSDBM
6. Use the srvctl utility to verify the status
y tu database.
s
ezsrvctlthstatus
i
[oracle@qr01db01 ~]$
h i @ e database -d dbm
Instance dbm1ois running
i r s t o us on node qr01db01
Instance
( j a sbdbm2nsise not running on node qr01db02
o hi l ice
[oracle@qr01db01 ~]$

i S that the DBM database is an administrator-managed Oracle RAC database created on


rNote
b
Jas
two database servers; qr01db01 and qr01db02. As mentioned previously, you will use
the database instance running on qr01db01 to perform most the practices. The other
database server (qr01db02) will not be started for most of the practices to avoid
unnecessary resource consumption in the laboratory environment.
7. Many practices refer to SQL scripts, which are provided as an alternative to typing lengthy
commands. These scripts are located in the labs directory under your student home
directory. List the files in the labs directory. Notice also the CSV, TTS and Exaconf
subdirectories. These subdirectories contain additional files for specific practices. You will
be directed to these files in the associated practice instructions.
[oracle@qr01db01 ~]$ cd labs
[oracle@qr01db01 labs]$ ls
CSV lab04-04-03.sql lab06-03-07.sql lab11-01-15.sql
Exaconf lab04-04-04.sql lab06-03-08.sql lab12-01-22.sql
lab03-02-03.sql lab06-02-03.sql lab06-04-21.sql lab15-01-33.sql
lab03-03-11.sql lab06-02-04.sql lab08-02-04.sql lab15-01-35.sql
lab03-03-12.sql lab06-02-06.sql lab08-02-06.sql lab15-01-36.sql
lab04-01-03.sql lab06-02-09.sql lab08-02-07.sql lab20-02-10.sql
lab04-01-04.sql lab06-02-18.sql lab08-02-15.sql lab20-02-11.sql

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 5
lab04-01-08.sql lab06-02-28.sql lab09-01-06.sql lab20-02-16.sql
lab04-02-03.sql lab06-02-38.sql lab09-01-07.sql lab20-03-13.sql
lab04-02-04.sql lab06-03-03.sql lab09-02-07.sql TTS
lab04-03-05.sql lab06-03-04.sql lab09-02-08.sql
lab04-03-08.sql lab06-03-05.sql lab09-02-09.sql
lab04-03-15.sql lab06-03-06.sql lab11-01-05.sql
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

[oracle@qr01db01 labs]$

8. Establish a terminal session connected to your first Exadata cell using the celladmin
user. Confirm that you are connected to the cell and then exit the session.
Note that you may see additional messages relating to server identities. Answer yes if you
are prompted to acknowledge server authenticity.
$ ssh celladmin@qr01cel01
celladmin@qr01cel01's password: <welcome>
[celladmin@qr01cel01 ~]$ cellcli -e list cell
ble
qr01cel01 online
fe r a
ans
[celladmin@qr01cel01 ~]$ exit
n - t r
no
logout
Connection to qr01cel01 closed. a
$
) has eฺ
9. Establish a terminal session connected to your secondc omExadata id using the
ucell
cellmonitor user. Confirm you are connected

chto the n
cell
G
t and then exit the session.
t e
fo related e
udto server identities. Answer yes if you
Note that you may see additional messages
yi n S t
e t h is
z authenticity.
are prompted to acknowledge server

h i@ use
$ ssh cellmonitor@qr01cel02
o
s b irs e to password: <welcome>
cellmonitor@qr01cel02's

( ja ens
[cellmonitor@qr01cel02
i
~]$ cellcli -e list cell
oh c
liqr01cel02 online
ir S
s b [cellmonitor@qr01cel02 ~]$ exit
Ja logout
Connection to qr01cel02 closed.
$
10. Exit your terminal sessions. We recommend that you start fresh terminal sessions at the
beginning of each practice, and that you exit all of your terminal sessions at the conclusion
of every practice. This eliminates the possibility that environment settings used in one
practice could cause problems in following practices.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 6
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
Practices )for s
haLesson
m i d eฺ 2:
Exadataฺ co Database
G u Machine:
h
c den
Overview
e t
o t
y i nf Chapter
S tu2
ez thi s
h i @ s e
o
irs e to u
s b
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 2: Exadata Database Machine: Overview


Chapter 2 - Page 1
Practices for Lesson 2
Practices Overview
There is no practice for Lesson 2.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 2: Exadata Database Machine: Overview


Chapter 2 - Page 2
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
Practices )for s
haLesson
m i d eฺ 3:
Exadataฺ co Database
G u Machine
h
c den
Architecture
e t
o t
y i nf Chapter
S tu3
ez thi s
h i @ s e
o
irs e to u
s b
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 1
Practices for Lesson 3
Practices Overview
In these practices, you will be familiarized with the Exadata cell architecture. You will:
• Examine the Exadata processes
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

• Exercise Exadata high availability


• Examine the hierarchy of cell objects
• Examine Exadata Smart Flash Cache

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 2
Practice 3-1: Process Familiarization
Overview
In this practice, you will examine the Exadata cell software processes.
Tasks
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

1. Establish a terminal connection to qr01cel01 as the celladmin user.


2. Restart Server (RS) is used to start up and shut down the Cell Server (CELLSRV) and
Management Server (MS). It also monitors these services to check whether they need to be
restarted. Locate the RS processes by using the following command:
[celladmin@qr01cel01 ~]$ ps -ef | grep cellrs
root 2025 1 0 18:29 ? 00:00:00
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellrssrm -ms 1 -
cellsrv 1
root 2032 2025 0 18:29 ? 00:00:00
ble
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellrsmmt -ms 1 -
fe r a
cellsrv 1
ans
root 2033 2025 0 18:29 ? 00:00:00
n - t r
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellrsbmt -ms 1 -
cellsrv 1 n o
a
has eฺ
root 2035 2033 0 18:29 ? 00:00:00
)
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellrsbkm -rs_conf

-ms_conf
ฺ c om Guid
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/deploy/config/cellinit.ora

t e ch dent
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/deploy/config/cellrsms.sta

nfo Stu
te -cellsrv_conf

yi
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/deploy/config/cellrsos.sta
z this
te -debug 0
e
root
o h i@ use
2036 2025 0 18:29 ? 00:00:01

irs e to
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellrsomt -ms 1 -
cellsrv 1
s b
i (
root
ja ens
2044 2035 0 18:29 ? 00:00:00

S oh lic
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellrssmt -rs_conf
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/deploy/config/cellinit.ora
ir
asb
-ms_conf
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/deploy/config/cellrsms.sta
J te -cellsrv_conf
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/deploy/config/cellrsos.sta
te -debug 0
1000 3609 3577 0 19:02 pts/0 00:00:00 grep cellrs
[celladmin@qr01cel01 ~]$

3. MS provides Exadata cell management and configuration. It works in cooperation with the
Exadata cell command-line interface (CellCLI). In addition, MS is responsible for sending
alerts and collects some statistics in addition to those collected by CELLSRV. Locate the MS
process by using the following command:
[celladmin@qr01cel01 ~]$ ps -ef | grep ms.err
root 2034 2032 1 18:29 ? 00:00:30 /usr/java/jdk1.5.0_15/bin/java
-Xms256m -Xmx512m -
Djava.library.path=/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib -
Ddisable.checkForUpdate=true -jar
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/oc4j/ms/j2ee/home/oc4j.jar -out
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/deploy/log/ms.lst -err
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/deploy/log/ms.err
1000 3623 3577 0 19:02 pts/0 00:00:00 grep ms.err
[celladmin@qr01cel01 ~]$
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 3
4. Locate the MS parent process. Use the parent process number associated with MS in the
output for step 3. Note that RS spawns (and, when required, re-spawns) MS.
[celladmin@qr01cel01 ~]$ ps -ef | grep 2032
root 2032 2025 0 18:29 ? 00:00:00
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellrsmmt -ms 1 -
cellsrv 1
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

root 2034 2032 1 18:29 ? 00:00:31 /usr/java/jdk1.5.0_15/bin/java


-Xms256m -Xmx512m -
Djava.library.path=/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib -
Ddisable.checkForUpdate=true -jar
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/oc4j/ms/j2ee/home/oc4j.jar -out
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/deploy/log/ms.lst -err
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/deploy/log/ms.err
1000 3625 3577 0 19:03 pts/0 00:00:00 grep 2032
[celladmin@qr01cel01 ~]$

5. CELLSRV is the primary Exadata software component and provides the majority of Exadata e
storage services. CELLSRV is a multithreaded server. Primarily, CELLSRV communicates r a bl
with Oracle Database to serve simple block requests, such as database buffer cache reads, s fe
and Smart Scan requests, such as table scans with projections and filters. CELLSRV also
- t r an
no
implements I/O Resource Management (IORM) and collects numerous statistics relating to n
a
its operations. Locate the CELLSRV process by using the following command:
s
[celladmin@qr01cel01 ~]$ ps -ef | grep "/cellsrv "ha
m ) i d eฺ
co t Gu
root 2037 2036 12 18:29 ? 00:04:28

c h ฺ
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellsrv
n
25 1200 9 5042
3652 3577 0 19:04 pts/0e 00:00:00
1000
f o t tude grep /cellsrv
[celladmin@qr01cel01 ~]$
z yin s S
6. Locate the CELLSRV parent process.
@ e Useththei parent process number associated with
h
CELLSRV in the output for
o i s
step 5. Note
u ethat RS spawns (and, when required, re-spawns)
CELLSRV.
s b irs e to
( ja ens ~]$ ps -ef | grep 2036
[celladmin@qr01cel01
i
S
h
root lic 2025 0 18:29 ?
o/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellrsomt
2036 00:00:01

b i r cellsrv 1 -ms 1 -

Jas root 2037 2036 13 18:29 ? 00:04:32


/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellsrv 25 1200 9 5042
1000 3654 3577 0 19:04 pts/0 00:00:00 grep 2036
[celladmin@qr01cel01 ~]$
7. Launch the Exadata cell command-line interface (CellCLI).
[celladmin@qr01cel01 ~]$ cellcli
CellCLI: Release 11.2.3.2.1 – Production...
CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 4
8. Execute the following CellCLI command to examine the attributes of the cell. Note that the
output also confirms that CELLSRV, MS, and RS are currently up and running. Exit CellCLI
after examining the cell attributes.
CellCLI> list cell detail
name: qr01cel01
bbuTempThreshold: 60
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

bbuChargeThreshold: 800
bmcType: absent
cellVersion: OSS_11.2.3.2.1_LINUX.X64_130109
cpuCount: 1
diagHistoryDays: 7
fanCount: 1/1
fanStatus: normal
flashCacheMode: WriteThrough
ble
id:
interconnectCount:
8ab50138-a667-4793-a976-c540dc1930c5
3
fe r a
ans
interconnect1:
iormBoost:
eth1
0.0
n - t r
n o
ipaddress1:
a 192.168.1.103/24

has eฺ
kernelVersion: 2.6.32-400.11.1.el5uek
)
om Guid
makeModel: Fake hardware
metricHistoryDays:
ฺ c 7
offloadEfficiency:
t e ch dent 663.5
powerCount:
yi nfo Stu 1/1
powerStatus:
e z this normal

o h i@ use
releaseVersion:
releaseTrackingBug:
11.2.3.2.1
14522699

s b irs e to
status: online

i ( ja ens
temperatureReading: 0.0

S oh lic
temperatureStatus: normal

ir upTime: 0 days, 0:36

Jasb cellsrvStatus:
msStatus:
running
running
rsStatus: running

CellCLI> exit
quitting

[celladmin@qr01cel01 ~]$
9. Exit all of your terminal sessions.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 5
Practice 3-2: Exadata High Availability
Overview
In this practice, you will observe some of the high-availability features of Exadata Storage
Server.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Tasks
1. Establish a terminal connection to qr01db01 as the oracle user.
2. Connect to your database with SQL*Plus. Log in as the sales user.
[oracle@qr01db01 ~]$ sqlplus sales/sales

SQL*Plus: Release 11.2.0.3.0 Production...

ble
SQL>
fe r a
3. Execute the SQL script /home/oracle/labs/lab03-02-03.sql. The script contains a
ans
series of I/O intensive queries, which are used in this practice to demonstrate how Oracle n - t r
no
Database is insulated from the different Exadata failure scenarios that are demonstrated in
a
has eฺ
the practice. Check on the workload periodically throughout the practice. If the workload
completes before you finish all the tasks, then simply re-execute the script to maintain an
)
active workload throughout the practice.
ฺ c om Guid
SQL> @/home/oracle/labs/lab03-02-03
t e ch dent
SQL> set timing on
i n fo Stu
SQL> select count(*) z
e y is
i@ use t h
from sales;

o h
...
s b irs e to
4. Establish (aja
i c e ns connection to the qr01cel01 Exadata cell as the root user.
separate terminal
h password
Theoroot li is oracle. Leave your SQL*Plus terminal session and workload
S
a s birLocate the process identification number for CELLSRV by using the following ps command.
running in the background.
J 5.
[root@qr01cel01 ~]# ps -ef | grep "/cellsrv "
root 2037 2036 12 18:29 ? 00:04:28
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellsrv 25 1200 9 5042
root 3752 3677 0 19:06 pts/0 00:00:00 grep /cellsrv
[root@qr01cel01 ~]#

6. Terminate the CELLSRV process by using the kill command and the process
identification number you observed in step 5.
[root@qr01cel01 ~]# kill -9 2037
[root@qr01cel01 ~]#

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 6
7. Re-execute the ps command from step 5. You should observe that CELLSRV is
automatically restarted with a new process identification number. How was CELLSRV
restarted?
[root@qr01cel01 ~]# ps -ef | grep "/cellsrv "
root 3801 3800 3 19:07 ? 00:00:00
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellsrv 25 1200 9 5042
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

root 3852 3735 0 19:07 pts/0 00:00:00 grep /cellsrv


[root@qr01cel01 ~]#
8. Check on the progress of your workload from step 3. You should observe that the workload
continues without error.
...
SQL> select count(*) from sales where amount_sold > 2;

COUNT(*) ble
---------- fe r a
ans
14693419
n - t r
a no
has eฺ
Elapsed: 00:00:25.72
)
SQL> select count(*) from sales where amount_sold > 3;

ฺ c om Guid
COUNT(*)
t e ch dent
----------
yi nfo Stu
14540531 e z this
o h i@ use
b irs e to
Elapsed: 00:00:18.79
s
SQL> jselect
i ( c ns
a ecount(*) from sales where amount_sold > 4;
h li
So
irOn qr01cel01, launch the Exadata cell command-line interface (CellCLI).
b
9.
Jas [root@qr01cel01 ~]# cellcli
CellCLI: Release 11.2.3.2.1 – Production...

CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 7
10. You have already seen how Exadata automatically recovers from an unexpected process
failure. Now observe the effect of restarting all the Exadata services.
CellCLI> alter cell restart services all

Stopping the RS, CELLSRV, and MS services...


Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

The SHUTDOWN of services was successful.


Starting the RS, CELLSRV, and MS services...
Getting the state of RS services... running
Starting CELLSRV services...
The STARTUP of CELLSRV services was successful.
Starting MS services...
The STARTUP of MS services was successful.
ble
CellCLI> fe r a
ans
11. Check on the progress of your workload from step 3. You should again observe that the
workload continues without error. n - t r
a no
has eฺ
...
)
SQL> select count(*) from sales where amount_sold > 7;

ฺ c om Guid
COUNT(*)
t e ch dent
----------
i n fo Stu
13929029 e y
z this
o h i@ use
s b irs e to
Elapsed: 00:00:24.37
SQL> jselect
i ( c ns
a ecount(*) from sales where amount_sold > 8;
h li
ir So
b COUNT(*)
Jas ----------
13776154

Elapsed: 00:00:18.29
SQL> select count(*) from sales where amount_sold > 9;

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 8
12. Exit your CellCLI session and re-execute the ps command from step 5. You should observe
that CELLSRV has a different process identification number because of the restart operation
executed in step 10.
CellCLI> exit
quitting
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

[root@qr01cel01 ~]# ps -ef | grep "/cellsrv "


root 4522 4519 9 19:13 ? 00:00:19
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/bin/cellsrv 25 1200 9 5042
root 4996 3735 0 19:16 pts/0 00:00:00 grep /cellsrv
[root@qr01cel01 ~]#

13. If the workload started in step 3 is still executing, stop it by typing <Control>-C in your
SQL*Plus session window. Wait until the workload stops.
...
ble
SQL> select count(*) from sales where amount_sold > 10;
fe r a
ans
COUNT(*) n - t r
----------
a no
13469722
) has eฺ
ฺ c om Guid
Elapsed: 00:00:18.05
e chwhere e n t
SQL> select count(*) from sales
f o t t u d amount_sold > 11;
i n
z* this S amount_sold > 11
^Cselect count(*) fromysales where
e
o h i@ use
ERROR at line
b s
iruser1:
to
a
ORA-01013:
j s n s erequested cancel of current operation
hi ( e
So lic
bir
Jas Elapsed: 00:00:09.30

SQL>
14. On qr01cel01, launch the Exadata cell command-line interface (CellCLI) again.
[root@qr01cel01 ~]# cellcli
CellCLI: Release 11.2.3.2.1 – Production...

CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 9
15. Execute the LIST ALERTHISTORY command. You should observe an alert indicating that
CELLSRV terminated unexpectedly. This alert relates to the process failure brought about
by killing CELLSRV in step 6. No alert appears for the controlled service restart executed in
step 10. If you don't see the alert, re-execute LIST ALERTHISTORY periodically until the
alert appears.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

CellCLI> list alerthistory


1_1 2013-07-17T18:31:57-04:00 warning
"Hugepage allocation failure in service cellsrv. Number of
Hugepages allocated is 0, failed to allocate 110"
2 2013-07-17T19:07:31-04:00 critical
"RS-7445 [Serv CELLSRV is absent] [It will be restarted] [] []
[] [] [] [] [] [] [] []"

CellCLI>
ble
fe
Note that the “Hugepage allocation failure” alert is peculiar to the laboratory environment. r a
You shouldn’t see this alert in a production environment. n s
16. Exit all of your terminal sessions. n - tra
a no
) has eฺ
ฺ c om Guid
t e ch dent
y i nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
a s bir
J

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 10
Practice 3-3: Storage Object Familiarization
Overview
In this practice, you are introduced to the hierarchy of Exadata storage objects.

Tasks
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

1. Establish a terminal connection to the qr01cel01 Exadata cell as the celladmin user.
2. Launch the Exadata cell command-line interface (CellCLI).
[celladmin@qr01cel01 ~]$ cellcli
CellCLI: Release 11.2.3.2.1 – Production...

CellCLI>
3. In Exadata, a LUN (Logical Unit) is a logical abstraction of a storage device. LUNs are
ble
based on hard disks and flash devices. LUNs are automatically created when Exadata is
fe r a
initially configured. Each Exadata cell contains 12 hard disk-based LUNs along with 16
ans
flash-based LUNs. List the LUNs on your primary Exadata cell.
n - t r
CellCLI> list lun
n o
a
has eฺ
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK00
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK00 normal
)
ฺ c om Guid
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK01

ch dent
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK01 normal

t e
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK02

yi nfo Stu
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK02 normal

z this
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK03
e
i@ use
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK03 normal

o h
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK04

s b irs e to
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK04 normal

i ( ja ens
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK05

lic
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK05 normal

S oh /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK06
ir
asb
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK06 normal

J /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK07
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK07 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK08
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK08 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK10
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK10 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK11
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK11 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH00
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH00 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH01
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH01 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH02
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH02 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH03
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH03 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH04
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH04 normal

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 11
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH05
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH05 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH06
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH06 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH07
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH07 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH08
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH08 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH09
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH09 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH10
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH10 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH11
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH11 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH12
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH12 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH13
ble
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH13 normal
fe r a
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH14
ans
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH14 normal
n - t r
no
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH15
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/FLASH15 normal
a
) has eฺ
m LUNs uidwith names and
CellCLI>
Note that the output from the virtualized Exadata ฺcell c oshows G
identifiers that are paths to virtualized diskse chvirtualized
and e n tflash devices. On a real Exadata
cell, the LUN names and identifiers are
t
o ontthe
fbased udPCI slot number and device number
y i n S
e z example,
of the hard disk or flash device. For
t h is here is the expected output for the LIST LUN
command on a real Exadata
CellCLI> list s o hi@ use
cell.

s b ir 0_0e to normal
lun

i ( ja 0_1 ens0_1 normal


0_0

S oh lic 0_2 normal


0_2
b ir
Jas
0_3 0_3 normal
0_4 0_4 normal
0_5 0_5 normal
0_6 0_6 normal
0_7 0_7 normal
0_8 0_8 normal
0_9 0_9 normal
0_10 0_10 normal
0_11 0_11 normal
1_0 1_0 normal
1_1 1_1 normal
1_2 1_2 normal
1_3 1_3 normal
2_0 2_0 normal
2_1 2_1 normal
2_2 2_2 normal
2_3 2_3 normal
4_0 4_0 normal
4_1 4_1 normal
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 12
4_2 4_2 normal
4_3 4_3 normal
5_0 5_0 normal
5_1 5_1 normal
5_2 5_2 normal
5_3 5_3 normal
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

4. List only the hard disk-based LUNs.


CellCLI> list lun where disktype = harddisk
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK00
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK00 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK01
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK01 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK02
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK02 normal
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK03
ble
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK03 normal
fe r a
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK04
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK04 normal
ans
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK05
n - t r
n
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK05 normal o
a
has eฺ
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK06
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK06 normal
)
ฺ om Guid
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK07
c
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK07 normal

t e ch dent
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK08

yi nfo Stu
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK08 normal

e z this
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09 normal

o h i@ use
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK10

s b irs e to
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK10 normal

i ( ja ens
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK11

lic
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK11 normal
oh
S CellCLI>
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 13
5. Examine the detailed attribute listing for the LUN whose name ends with DISK09. Note the
attribute setting isSystemLun=FALSE. This indicates that the LUN is not located on a
system disk. Notice also that the LUN is associated with one physical disk and one cell
disk.
CellCLI> list lun where name like '.*DISK09' detail
name:
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09
cellDisk: CD_09_qr01cel01
deviceName:
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09
diskType: HardDisk
id:
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09
isSystemLun: FALSE
lunAutoCreate: FALSE
ble
lunSize: 11
fe r a
physicalDrives:
ans
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09
n - t r
no
raidLevel: "RAID 0"
status: normal
a
) has eฺ
CellCLI>
c om au
id
6. Exadata maintains the physical attributes of eachฺhard
chfor eachn t G
disk in physicaldisk object. A
t
physicaldisk object is automatically created e e hard disk. Examine the attributes
yi
for the hard disk associated with LUN
S tudin the previous step.
fo examined
nyou
CellCLI> list physicaldiske is like '.*DISK09' detail
z wherethluns
name:
o h i@ use
s b irs e to
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09

( j a diskType:
n s HardDisk

o hi luns: l ice
rS
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09

s b i physicalInsertTime: 2013-02-24T21:12:47-05:00

Ja physicalSize:
status:
11
normal

CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 14
7. A cell disk is a higher-level storage abstraction. Each cell disk is based on a LUN and
contains additional attributes and metadata. Examine the attributes for the cell disk-based
on the LUN you examined in step 5.
CellCLI> list celldisk CD_09_qr01cel01 detail
name: CD_09_qr01cel01
comment:
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

creationTime: 2013-02-28T16:31:52-05:00
deviceName:
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09
devicePartition:
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09
diskType: HardDisk
errorCount: 0
freeSpace: 0
id: ed8be9ac-5851-4e03-aacb-c5eec33156fa
ble
interleaving: none
fe r a
lun:
ans
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09
n - t r
physicalDisk:
n o
a
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK09

has eฺ
raidLevel: "RAID 0"
)
om Guid
size: 1.5625G
status:
ฺ c
normal
h ent
t e c
CellCLI>
yi nfo Stud
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 15
8. A grid disk defines an area of storage on a cell disk. Grid disks are consumed by ASM and
are used as the storage for ASM disk groups. Each cell disk can contain a number of grid
disks. Examine the grid disks associated with the cell disk you examined in the previous
step. Note the names and sizes of the grid disks.
CellCLI> list griddisk where celldisk=CD_09_qr01cel01 detail
name: DATA_QR01_CD_09_qr01cel01
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

asmDiskgroupName: DATA_QR01
asmDiskName: DATA_QR01_CD_09_QR01CEL01
asmFailGroupName: QR01CEL01
availableTo:
cachingPolicy: default
cellDisk: CD_09_qr01cel01
comment:
creationTime: 2013-02-28T16:32:10-05:00
diskType: HardDisk
ble
errorCount: 0
fe r a
id: 9a735142-8a0f-4798-a83e-814ceb7f9ab5
ans
offset: 208M
n - t r
size: 592M n o
a
has eฺ
status: active
)
m uid
name:
c o
DBFS_DG_CD_09_qr01cel01
asmDiskgroupName:
c h ent G
DBFS_DG
e

asmDiskName: t tud
foDBFS_DG_CD_09_QR01CEL01
asmFailGroupName: yin QR01CEL01
availableTo: e
z this S
o h i@ use default
cachingPolicy:

s b irs e to
cellDisk: CD_09_qr01cel01

i ( ja comment:
c e ns
S oh li
creationTime: 2013-02-28T16:32:03-05:00

ir diskType: HardDisk

Jasb errorCount:
id:
0
dfb7dcda-f9fa-404b-9239-0f03e3cea480
offset: 48M
size: 160M
status: active

name: RECO_QR01_CD_09_qr01cel01
asmDiskgroupName: RECO_QR01
asmDiskName: RECO_QR01_CD_09_QR01CEL01
asmFailGroupName: QR01CEL01
availableTo:
cachingPolicy: default
cellDisk: CD_09_qr01cel01
comment:
creationTime: 2013-02-28T16:32:24-05:00
diskType: HardDisk
errorCount: 0
id: 7e028f52-604d-4189-8c2c-dafcd01b8f17

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 16
offset: 800M
size: 800M
status: active

CellCLI>
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

9. Establish a terminal connection to qr01db01 as the grid user.


10. Using SQL*Plus, connect to ASM as sysasm.
[grid@qr01db01 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production...

SQL>
ble
11. Locate the grid disks from step 8 inside ASM (use the SQL script
fe r a
/home/oracle/labs/lab03-03-11.sql if you prefer). Check that the sizes reported
ans
by ASM match the grid disk attributes reported in step 8. Note the capitalization of the value
n - t r
in the like string.
a no
has eฺ
SQL> select name, path, state, total_mb from v$asm_disk
)
om Guid
2> where name like '%_CD_09_QR01CEL01';
ฺ c
t e ch dent
nfo Stu
NAME
y
------------------------------i
z this
e
PATH
o h i@ use
b s
--------------------------------------------------------------
irTOTAL_MB
to
STATE
s ns
ja ---------- e
i (
--------
h c e
li
oRECO_QR01_CD_09_QR01CEL01
S o/192.168.1.103/RECO_QR01_CD_09_qr01cel01
ir
Jasb NORMAL 800

DATA_QR01_CD_09_QR01CEL01
o/192.168.1.103/DATA_QR01_CD_09_qr01cel01
NORMAL 592

DBFS_DG_CD_09_QR01CEL01
o/192.168.1.103/DBFS_DG_CD_09_qr01cel01
NORMAL 160

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 17
12. Determine which ASM disk group the grid disks from step 8 are assigned to (use the SQL
script /home/oracle/labs/lab03-03-12.sql if you prefer). Note the capitalization of
the value in the like string.
SQL> select d.name disk, dg.name diskgroup
2> from v$asm_disk d, v$asm_diskgroup dg
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

3> where dg.group_number = d.group_number


4> and d.name like '%_CD_09_QR01CEL01';

DISK DISKGROUP
------------------------------ ------------------------------
RECO_QR01_CD_09_QR01CEL01 RECO_QR01
DATA_QR01_CD_09_QR01CEL01 DATA_QR01
DBFS_DG_CD_09_QR01CEL01 DBFS_DG
ble
fe r a
ans
SQL>
n - t r
13. Exit all your SQL*Plus and CellCLI sessions.
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 18
Practice 3-4: Exadata Smart Flash Cache Familiarization
Overview
In this practice, you are introduced to Exadata Smart Flash Cache.

Tasks
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

1. Establish a terminal connection to the qr01cel01 Exadata cell as the celladmin user.
2. Launch the Exadata cell command-line interface (CellCLI).
[celladmin@qr01cel01 ~] $ cellcli
CellCLI: Release 11.2.3.2.1 – Production...

CellCLI>
3. List the cell disks associated with the flash disk modules in your Exadata cell. By default,
ble
there should be 16 cell disks having names that start with FD.
fe r a
CellCLI> list celldisk where disktype=flashdisk ans
n - t r
FD_00_qr01cel01 normal
n o
FD_01_qr01cel01 normal a
FD_02_qr01cel01 normal
) has eฺ
FD_03_qr01cel01 normal
ฺ c om Guid
FD_04_qr01cel01
t e ch dent
normal
FD_05_qr01cel01
yi nfo Stunormal

e z this
FD_06_qr01cel01 normal

o h i@ use
FD_07_qr01cel01 normal

s b irs e to
FD_08_qr01cel01 normal

i ( ja ens
FD_09_qr01cel01 normal

S oh lic
FD_10_qr01cel01 normal
ir FD_11_qr01cel01 normal
Jasb FD_12_qr01cel01 normal
FD_13_qr01cel01 normal
FD_14_qr01cel01 normal
FD_15_qr01cel01 normal

CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 19
4. By default, Exadata Smart Flash Cache is configured across all the flash-based cell disks.
Use the LIST FLASHCACHE DETAIL command to confirm that Exadata Smart Flash
Cache is configured on your flash-based cell disks. Note that the size of the Exadata Smart
Flash Cache on your laboratory cells is much smaller than what you would observe on a
real cell; however, all of the other attributes would be similar on a real cell.
CellCLI> list flashcache detail
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

name: qr01cel01_FLASHCACHE
cellDisk:
FD_01_qr01cel01,FD_13_qr01cel01,FD_07_qr01cel01,FD_06_qr01cel01,FD_09_qr01cel0
1,FD_11_qr01cel01,FD_02_qr01cel01,FD_03_qr01cel01,FD_05_qr01cel01,FD_04_qr01ce
l01,FD_15_qr01cel01,FD_14_qr01cel01,FD_08_qr01cel01,FD_00_qr01cel01,FD_10_qr01
cel01,FD_12_qr01cel01
creationTime: 2013-03-08T02:18:12-05:00
degradedCelldisks:
effectiveCacheSize: 3G
ble
id:
size:
c0dca501-f09a-46f0-b504-ec26f23def79
3G
fe r a
ans
status: normal

n - t r
CellCLI>
a no
5. In addition to Exadata Smart Flash Cache, Exadata Smart Flash
) h asLog provides
ฺ a

Flash Log uses a small portion of high-performance c o m memory


flash u idasetemporary
mechanism for improving the latency of database redo log write operations. Exadata Smart

c
facilitate low latency redo log writes. By default, h ฺ
ExadatanSmartt G Flash Log uses 32storage
MB on
to

o
each flash-based cell disk, for a total off512 teMB ontueach
de Exadata Storage Server. Use the
i n S the Exadata Smart Flash Log area on this
y to examine
LIST FLASHLOG DETAIL command
e z h i s
cell.
h i @ se t
r
CellCLI> list flashlog
i s o todetail
u
a b
scellDisk:
name:
s e qr01cel01_FLASHLOG
( j n
o hi l i ce
FD_08_qr01cel01,FD_10_qr01cel01,FD_04_qr01cel01,FD_13_qr01cel01,FD_09_qr01cel0

b i r S 1,FD_11_qr01cel01,FD_00_qr01cel01,FD_05_qr01cel01,FD_14_qr01cel01,FD_02_qr01ce
Jas
l01,FD_15_qr01cel01,FD_03_qr01cel01,FD_06_qr01cel01,FD_01_qr01cel01,FD_12_qr01
cel01,FD_07_qr01cel01
creationTime: 2013-02-28T16:21:30-05:00
degradedCelldisks:
effectiveSize: 512M
efficiency: 100.0
id: 1417f53b-49c3-4419-919c-933b812f0159
size: 512M
status: normal

CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 20
6. Use the LIST FLASHCACHECONTENT DETAIL command to show information about the
data inside Exadata Smart Flash Cache. You can see that each entry contains a series of
attributes relating to a database object in the cache. For each object, you can see how
much data is being cached along with the number of cache hits and misses. This
information can help you to assess cache efficiency for specific database objects.
CellCLI> list flashcachecontent detail
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

...
cachedKeepSize: 0
cachedSize: 8192
dbID: 2080757153
dbUniqueName: DBM
hitCount: 0
missCount: 0
ble
objectNumber: 290
fe r a
tableSpaceNumber: 0
ans
n - t r
cachedKeepSize: n 0 o
a
cachedSize:
) has eฺ 73728

om Guid
dbID: 2080757153
ฺ c
ch dent
dbUniqueName: DBM
t e
nfo Stu
hitCount: 0
missCount: yi
z this 0
e
o h i@ use
objectNumber: 457

irs e to
tableSpaceNumber: 0
s b
i ( c e ns
ja cachedKeepSize: 0
S oh licachedSize: 24576
ir
Jasb dbID:
dbUniqueName:
2080757153
DBM
hitCount: 0
missCount: 0
objectNumber: 458
tableSpaceNumber: 0

cachedKeepSize: 0
cachedSize: 8192
dbID: 2080757153
dbUniqueName: DBM
hitCount: 0
missCount: 0
objectNumber: 461
tableSpaceNumber: 0

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 21
cachedKeepSize: 0
cachedSize: 65536
dbID: 2080757153
dbUniqueName: DBM
hitCount: 101
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

missCount: 1
objectNumber: 4294967294
tableSpaceNumber: 0

CellCLI>
7. Exit your CellCLI session.

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Exadata Database Machine Architecture


Chapter 3 - Page 22
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
s
haLesson
Practices )for
m i d eฺ 4: Key
Capabilities
ฺ co t Gofu Exadata
h
c denMachine
Database
e
o t
y i nf Chapter
S tu4
ez thi s
h i @ s e
o
irs e to u
s b
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 1
Practices for Lesson 4
Practices Overview
In these practices, you are introduced to four major capabilities of Exadata, namely:
• Smart Scan
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

• Exadata Hybrid Columnar Compression


• Exadata Smart Flash Cache
• Storage Index

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 2
Practice 4-1: Smart Scan
Overview
In this practice, you are introduced to the Smart Scan capability of Exadata. You will execute a
query with and without Smart Scan enabled and you will examine statistics to measure the
effect of Smart Scan.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Tasks
1. Establish a terminal connection to qr01db01 as the oracle user.
2. Connect to your database with SQL*Plus. Log in as the sales user.
[oracle@qr01db01 ~]$ sqlplus sales/sales

SQL*Plus: Release 11.2.0.3.0 Production...


ble
fe r a
SQL>
t r a ns
n-
3. Execute the following query (or execute the SQL script /home/oracle/labs/lab04-
01-03.sql) and verify that the statistics are at or near zero values: no
s a
SQL> select a.name, b.value/1024/1024 MB
a
) h ideฺ
2 from v$sysstat a, v$mystat b
c m
o Gu
3 where a.statistic# = b.statistic# and
4 (a.name in ('physical read totalcbytes', nt h ฺ
f o te bytes',
t u de
5 'physical write
yi n total
Sbytes')
6
z
'cell IO uncompressed
e h i s
i @ se t
7 or a.name like 'cell phy%');
h
b i rso e to u
NAME
a s
(j read n s
MB

h i l i c etotal
---------------------------------------------------------------- ----------

S ophysical write total bytes


physical bytes .015625

s b ir cell physical IO interconnect bytes 0

Ja cell physical IO bytes saved during optimized file creation


.015625
0
cell physical IO bytes saved during optimized RMAN file restore 0
cell physical IO bytes eligible for predicate offload 0
cell physical IO bytes saved by storage index 0
cell physical IO bytes sent directly to DB node to balance CPU 0
cell physical IO interconnect bytes returned by smart scan 0
cell IO uncompressed bytes 0

10 rows selected.

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 3
4. Execute the following query (or execute the SQL script /home/oracle/labs/lab04-
01-04.sql). Note the optimizer hint that disables Smart Scan for the query.
SQL> select /*+ OPT_PARAM('cell_offload_processing' 'false') */
2 count(*) from sales
3 where time_id between '01-JAN-2003' and '31-DEC-2003'
4 and amount_sold = 1;
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

COUNT(*)
----------
10088

SQL>
5. Repeat the statistics query from step 3 (or execute the SQL script
/home/oracle/labs/lab04-01-03.sql). Note that all of the data processed by the
ble
query in step 4 (physical read total bytes) is returned to the database server over
fe r a
the storage network (cell physical IO interconnect bytes).
ans
SQL> select a.name, b.value/1024/1024 MB
n - t r
2 from v$sysstat a, v$mystat b
a no
has eฺ
3 where a.statistic# = b.statistic# and
4 (a.name in ('physical read total bytes',
)
5
om Guid
'physical write total bytes',
ฺ c
6
ch dent
'cell IO uncompressed bytes')
t e
nfo Stu
7 or a.name like 'cell phy%');

yi
z this
NAME
@ e MB
i
h bytes us e
----------------------------------------------------------------
o ----------
physical readrs o
physical s b i total
t
e bytes
559.054688

( j a e n sinterconnect
write total 0

hi
cell
lic IO bytes saved during optimized file creation
physical IO bytes 559.054688

ir Socell physical 0
b
Jas
cell physical IO bytes saved during optimized RMAN file restore 0
cell physical IO bytes eligible for predicate offload 0
cell physical IO bytes saved by storage index 0
cell physical IO bytes sent directly to DB node to balance CPU 0
cell physical IO interconnect bytes returned by smart scan 0
cell IO uncompressed bytes 0

10 rows selected.

SQL>
6. Reconnect to your database in order to reset the session level statistics.
SQL> connect sales/sales
Connected.
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 4
7. Repeat the statistics query from step 3 (or execute the SQL script
/home/oracle/labs/lab04-01-03.sql) and verify that the statistics are again at or
near zero values:
SQL> select a.name, b.value/1024/1024 MB
2 from v$sysstat a, v$mystat b
3 where a.statistic# = b.statistic# and
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

4 (a.name in ('physical read total bytes',


5 'physical write total bytes',
6 'cell IO uncompressed bytes')
7 or a.name like 'cell phy%');

NAME MB
---------------------------------------------------------------- ----------
physical read total bytes 0
ble
physical write total bytes
cell physical IO interconnect bytes
0
0
fe r a
ans
cell physical IO bytes saved during optimized file creation
cell physical IO bytes saved during optimized RMAN file restore
0
0
n - t r
cell physical IO bytes eligible for predicate offload
a no 0

has eฺ
cell physical IO bytes saved by storage index 0
)
om Guid
cell physical IO bytes sent directly to DB node to balance CPU 0

ฺ c
cell physical IO interconnect bytes returned by smart scan 0
cell IO uncompressed bytes
t e ch dent 0

y i nfo Stu
10 rows selected.
e z this
SQL>
o h i@ use
s b irs query
8. Execute the following e t(oro execute the SQL script /home/oracle/labs/lab04-
i (
01-08.sql). ja This isenthessame query as in step 4; however, this time there is no optimizer
hinto
S lic Scan.
tohdisable Smart

a s bir SQL> select count(*) from sales


J 2 where time_id between '01-JAN-2003' and '31-DEC-2003'
3 and amount_sold = 1;

COUNT(*)
----------
10088

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 5
9. Repeat the statistics query from step 3 (or execute the SQL script
/home/oracle/labs/lab04-01-03.sql). Note that the query still performs
approximately 559 MB of I/O (physical read total bytes). However, this time only
about 228 KB is actually returned to the database server (cell physical IO
interconnect bytes). This is Smart Scan in action.
Also note that in this case, Smart Scan is acting on all of the I/O associated with the query.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

This is the case because cell physical IO bytes eligible for predicate
offload equals physical read total bytes, and cell physical IO
interconnect bytes returned by smart scan equals cell physical IO
interconnect bytes.
SQL> select a.name, b.value/1024/1024 MB
2 from v$sysstat a, v$mystat b
3 where a.statistic# = b.statistic# and
4 (a.name in ('physical read total bytes',
ble
5 'physical write total bytes',
fe r a
6 'cell IO uncompressed bytes')
ans
7 or a.name like 'cell phy%');
n - t r
n o
NAME a MB

) has eฺ
---------------------------------------------------------------- ----------
physical read total bytes
ฺ c om Guid 559.039063

ch dent
physical write total bytes 0

t e
cell physical IO interconnect bytes .222244263

i nfo Stu
cell physical IO bytes saved during optimized file creation
y 0

e z this
cell physical IO bytes saved during optimized RMAN file restore 0

h i@ use
cell physical IO bytes eligible for predicate offload
o
559.039063

s b irs e to
cell physical IO bytes saved by storage index 0

ja ens
cell physical IO bytes sent directly to DB node to balance CPU 0
i (
oh lic
cell physical IO interconnect bytes returned by smart scan .222244263

ir S cell IO uncompressed bytes 559.039063

Jasb 10 rows selected.

SQL>
10. Exit your SQL*Plus session.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 6
Practice 4-2: Exadata Hybrid Columnar Compression
Overview
In this practice, you are introduced to Exadata Hybrid Columnar Compression. You will create
compressed copies of an existing database table and examine the level of compression you
achieve.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Tasks
1. Establish a terminal connection to qr01db01 as the oracle user.
2. Connect to your database with SQL*Plus. Log in as the sales user.
[oracle@qr01db01 ~]$ sqlplus sales/sales

SQL*Plus: Release 11.2.0.3.0 Production...


ble
fe r a
SQL>
t r a ns
3. Determine the size of the uncompressed MYCUSTOMERS table (use the SQL -script
/home/oracle/labs/lab04-02-03.sql if you prefer). n on
s a
SQL> col segment_name format a30 a
) h ideMBฺ
SQL> select segment_name, sum(bytes)/1024/1024 m
co t Gu
2 from user_segments h ฺ
c den
3 where segment_name like o t e
'MYCUST%'
y i nf S tu
ez thi
4 group by segment_name;
s
i @ se
SEGMENT_NAMEsoh u MB
i r t o
( j a sb nse
------------------------------ ----------

o hi
MYCUSTOMERS
l ice 208

b i r S SQL>
Jas 4. Verify that the CUSTOMERS table is uncompressed (use the SQL script
/home/oracle/labs/lab04-02-04.sql if you prefer).
SQL> select table_name, compression, compress_for
2 from user_tables
3 where table_name like 'MYCUST%';

TABLE_NAME COMPRESS COMPRESS_FOR


------------------------------ -------- ------------
MYCUSTOMERS DISABLED

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 7
5. Exadata Hybrid Columnar Compression achieves its highest levels of compression with
data that is direct-path inserted. Execute the following ALTER SESSION commands to
ensure the use of direct-path inserts later in the practice.
SQL> alter session force parallel query;
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Session altered.

SQL> alter session force parallel ddl;

Session altered.

SQL> alter session force parallel dml;

ble
Session altered.
fe r a
ans
SQL>
n - t r
6. a no
Create a compressed copy of the MYCUSTOMERS table by using the QUERY HIGH
warehouse compression mode.
) has eฺ
SQL> create table mycust_query compress
c o id high
m for uquery
2 parallel 4 nologging as select ฺ
ch d*efrom G
nt mycustomers;
t e
y i nfo Stu
Table created.
e z this
o h i@ use
SQL>
s b irs e to
jamode.eNote
7. Create a compressed
( s of the MYCUSTOMERS table using the ARCHIVE HIGH archive
ncopy
oh i
compression li c that it may take approximately one minute for the table to be
S
created.
a s bir SQL> create table mycust_archive compress for archive high
J 2 parallel 4 nologging as select * from mycustomers;

Table created.

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 8
8. Verify the compression mode settings for the tables you just created (use the SQL script
/home/oracle/labs/lab04-02-04.sql again if you prefer).
SQL> select table_name, compression, compress_for
2 from user_tables
3 where table_name like 'MYCUST%';
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

TABLE_NAME COMPRESS COMPRESS_FOR


------------------------------ -------- ------------
MYCUST_QUERY ENABLED QUERY HIGH
MYCUST_ARCHIVE ENABLED ARCHIVE HIGH
MYCUSTOMERS DISABLED

SQL> bl e
r a
created (use the SQL script /home/oracle/labs/lab04-02-03.sql if youra n se
9. Compare the size of the original uncompressed table with the two compressed copiesfyou
prefer).
Calculate the compression ratios achieved using the formula:
o n -t
Compression Ratio = Uncompressed Size / CompressedaSize n
SQL> select segment_name,sum(bytes)/1024/1024
) has MBeฺ
2 from user_segments
ฺ c om Guid
3 where segment_name like 'MYCUST%'
t e ch dent
4 group by segment_name;
yi nfo Stu
e z this
SEGMENT_NAME
o h i@ use MB

s b irs e to
------------------------------ ----------

( ja ens
MYCUSTOMERS
i
208
h li
oMYCUST_ARCHIVE c 18
ir S
s b MYCUST_QUERY 31
Ja
SQL>
10. Drop the compressed tables that you created in this practice.
SQL> drop table mycust_query;

Table dropped.

SQL> drop table mycust_archive;

Table dropped.

SQL>
11. Exit your SQL*Plus session.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 9
Practice 4-3: Exadata Smart Flash Cache
Overview
In this practice, you will examine the use of Exadata Smart Flash Cache. You will execute a
series of record lookups and use database statistics to verify the use of Exadata Smart Flash
Cache. You will also compare the execution statistics with and without the use of Exadata Smart
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Flash Cache.

Tasks
1. Establish a terminal connection to the qr01cel01 Exadata cell as the celladmin user.
2. Execute the following two commands to drop and then re-create Exadata Smart Flash
Cache on all of your Exadata cells. You must perform this action so that Exadata Smart
Flash Cache is empty at the beginning of this practice; thus ensuring consistent results later
in the practice. To do this, you will use the distributed command line utility (dcli) that is
ble
provided with Exadata. Using dcli you can execute cell-level administrative commands
fe r a
simultaneously on multiple Exadata cells. A more detailed discussion of dcli features and
ans
options is provided later in the course. Be careful not to add any extra spaces in the server
n - t r
list following the dcli –c command-line option.
n o
a
has eฺ
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
> drop flashcache
)
ฺ c om Guid
qr01cel01: Flash cache qr01cel01_FLASHCACHE successfully dropped

t e ch dent
qr01cel02: Flash cache qr01cel02_FLASHCACHE successfully dropped

nfo Stu
qr01cel03: Flash cache qr01cel03_FLASHCACHE successfully dropped

yi
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
z this
e
> create flashcache all

h i@ use
qr01cel01: Flash cache qr01cel01_FLASHCACHE successfully created
o
s b irs e to
qr01cel02: Flash cache qr01cel02_FLASHCACHE successfully created

i ( ja ens
qr01cel03: Flash cache qr01cel03_FLASHCACHE successfully created

S oh lic
[celladmin@qr01cel01 ~]$

3.
b i rEstablish a separate terminal connection to qr01db01 as the oracle user.

J s
a 4. Connect to your database with SQL*Plus. Log in as the sales user.
[oracle@qr01db01 ~]$ sqlplus sales/sales

SQL*Plus: Release 11.2.0.3.0 Production...

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 10
5. Execute the following query (or execute the SQL script /home/oracle/labs/lab04-
03-05.sql) and verify that the statistics are at or near zero values:
SQL> select a.name, b.value from v$sysstat a, v$mystat b
2 where a.statistic# = b.statistic# and
3 (a.name like '%flash cache read hits'
4 or a.name like 'cell phy%'
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

5 or a.name like 'physical read tot%'


6 or a.name like 'physical read req%');

NAME VALUE
---------------------------------------------------------------- ----------
physical read total IO requests 0
physical read total multi block requests 0
physical read requests optimized 0
ble
physical read total bytes optimized 0
fe r a
physical read total bytes 0
ans
cell physical IO interconnect bytes 0
n - t r
no
cell physical IO bytes saved during optimized file creation 0
cell physical IO bytes saved during optimized RMAN file restore
a 0
cell physical IO bytes eligible for predicate offload
) has eฺ 0

om Guid
cell physical IO bytes saved by storage index 0

ฺ c
cell physical IO bytes sent directly to DB node to balance CPU 0

e ch dent
cell physical IO interconnect bytes returned by smart scan
t 0
cell flash cache read hits
y i nfo Stu 0

e z this
13 rows selected.
o h i@ use
SQL>
s b irs e to
6. Flush thei ( ja cache
buffer
c e ntos ensure that the queries in step 8 must retrieve the required data
h Exadatali cells.
fromothe
S
a s bir SQL> alter system flush buffer_cache;
J
System altered.

SQL>
7. Configure the session to display server output.
SQL> set serveroutput on
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 11
8. The following PL/SQL block performs 500 record lookups spread across a reasonably large
table. The workload is representative of the scattered record access normally associated
with an OLTP application. Execute the PL/SQL block against your database (or execute the
SQL script /home/oracle/labs/lab04-03-08.sql). Note that the workload may take
a few minutes to complete.
SQL> declare
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

2 a number;
3 s number := 0;
4 begin
5 for n in 1 .. 500 loop
6 select cust_credit_limit into a from customers
7 where cust_id=n*2000;
8 s := s+a;
ble
9 end loop;
fe r a
10 dbms_output.put_line('Transaction total = '||s);
ans
11 end;
n - t r
12 / n o
a
Transaction total = 3761500
) has eฺ
ฺ c om Guid
ch dent
PL/SQL procedure successfully completed.
t e
SQL> yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
So
h lic
b ir
Jas

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 12
9. Repeat the statistics query from step 5 (or execute the SQL script
/home/oracle/labs/lab04-03-05.sql). Note the high number of IO requests
(physical read total IO requests) relative to the low number of optimized
requests (physical read requests optimized and cell flash cache read
hits). This indicates that the queries were mostly satisfied by using physical disk reads
and is indicative of a recently emptied cache.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

SQL> select a.name, b.value from v$sysstat a, v$mystat b


2 where a.statistic# = b.statistic# and
3 (a.name like '%flash cache read hits'
4 or a.name like 'cell phy%'
5 or a.name like 'physical read tot%'
6 or a.name like 'physical read req%');

NAME VALUE
ble
---------------------------------------------------------------- ----------
fe r a
physical read total IO requests 959
ans
physical read total multi block requests 46
n - t r
physical read requests optimized
n
140
o
physical read total bytes optimized
a 1146880
physical read total bytes
) has eฺ 17833984

om Guid
cell physical IO interconnect bytes 17833984

ฺ c
cell physical IO bytes saved during optimized file creation 0

e ch dent
cell physical IO bytes saved during optimized RMAN file restore
t 0

yi nfo Stu
cell physical IO bytes eligible for predicate offload 0

e z this
cell physical IO bytes saved by storage index 0

o h i@ use
cell physical IO bytes sent directly to DB node to balance CPU 0

irs e to
cell physical IO interconnect bytes returned by smart scan 0

s b
cell flash cache read hits 140

i ( ja ens
S lic
o13h rows selected.
ir
Jasb SQL>
10. Reconnect to your database in order to reset the session level statistics.
SQL> connect sales/sales
Connected.
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 13
11. Repeat the statistics query from step 5 (or execute the SQL script
/home/oracle/labs/lab04-03-05.sql) and verify that the statistics are again at or
near zero values:
SQL> select a.name, b.value from v$sysstat a, v$mystat b
2 where a.statistic# = b.statistic# and
3 (a.name like '%flash cache read hits'
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

4 or a.name like 'cell phy%'


5 or a.name like 'physical read tot%'
6 or a.name like 'physical read req%');

NAME VALUE
---------------------------------------------------------------- ----------
physical read total IO requests 1
physical read total multi block requests 0
ble
physical read requests optimized
physical read total bytes optimized
0
0
fe r a
ans
physical read total bytes
cell physical IO interconnect bytes
8192
8192
n - t r
cell physical IO bytes saved during optimized file creation
a no 0

has eฺ
cell physical IO bytes saved during optimized RMAN file restore 0
)
om Guid
cell physical IO bytes eligible for predicate offload 0
cell physical IO bytes saved by storage index
ฺ c 0

e ch dent
cell physical IO bytes sent directly to DB node to balance CPU
t
0

y i nfo Stu
cell physical IO interconnect bytes returned by smart scan 0
cell flash cache read hits
e z this 0

13 rows selected. h
o i@ use
s b irs e to
SQL> ja
i ( c e ns
ohflush thelbuffer
12. Again,
S
i cache to ensure that the queries in the next step must retrieve the
a s birrequired
output.
data from the Exadata cells. Also, configure your new session to display server
J SQL> alter system flush buffer_cache;

System altered.

SQL> set serveroutput on


SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 14
13. Re-execute the PL/SQL block introduced in step 8 (or execute the SQL script
/home/oracle/labs/lab04-03-08.sql). Confirm that the result is the same.
SQL> declare
2 a number;
3 s number := 0;
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

4 begin
5 for n in 1 .. 500 loop
6 select cust_credit_limit into a from customers
7 where cust_id=n*2000;
8 s := s+a;
9 end loop;
10 dbms_output.put_line('Transaction total = '||s);
11 end;
ble
12 / fe r a
ans
Transaction total = 3761500
n - t r
n o
a
has eฺ
PL/SQL procedure successfully completed.

)
SQL>
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 15
14. Repeat the statistics query from step 5 (or execute the SQL script
/home/oracle/labs/lab04-03-05.sql). Compare the values for cell flash
cache read hits and physical read total IO requests. They should be much
closer together, indicating that most of the I/Os were satisfied by Exadata Smart Flash
Cache.
SQL> select a.name, b.value from v$sysstat a, v$mystat b
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

2 where a.statistic# = b.statistic# and


3 (a.name like '%flash cache read hits'
4 or a.name like 'cell phy%'
5 or a.name like 'physical read tot%'
6 or a.name like 'physical read req%');

NAME VALUE
---------------------------------------------------------------- ----------
physical read total IO requests 1013
ble
physical read total multi block requests 0
fe r a
physical read requests optimized 905
ans
physical read total bytes optimized 7413760
n - t r
physical read total bytes
n 8298496 o
a
has eฺ
cell physical IO interconnect bytes 8298496

)
cell physical IO bytes saved during optimized file creation 0

ฺ c om Guid
cell physical IO bytes saved during optimized RMAN file restore 0

t e ch dent
cell physical IO bytes eligible for predicate offload 0

nfo Stu
cell physical IO bytes saved by storage index 0

yi
cell physical IO bytes sent directly to DB node to balance CPU
z this 0
e
cell physical IO interconnect bytes returned by smart scan 0

o h i@ use
cell flash cache read hits 905

s birs se to
13 rowsaselected.
h i (j icen
S oSQL> l
ir
Jasb
In an earlier practice, you saw how to obtain general information about Exadata Smart Flash
Cache on an Exadata cell using the LIST FLASHCACHECONTENT CellCLI command. Over the
remainder of this practice, you will learn how to isolate specific information in Exadata Smart
Flash Cache.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 16
15. Use the following query (or execute the SQL script /home/oracle/labs/lab04-03-
15.sql) to determine the number of optimized physical reads (reads optimized by Exadata
Smart Flash Cache or Exadata storage index) for the SALES.CUSTOMERS table. Note the
tablespace number (TS#) and object number (DATAOBJ#) associated with the table.
SQL> select owner, object_name, tablespace_name, ts#, dataobj#,
2 statistic_name, value
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

3 from v$segment_statistics
4 where owner='SALES' and object_name='CUSTOMERS'
5 and statistic_name='optimized physical reads';

OWNER OBJECT_NAME
------------------------------ ------------------------------
TABLESPACE_NAME TS# DATAOBJ#
------------------------------ ---------- ----------
STATISTIC_NAME VALUE
ble
---------------------------------------------------------------- ----------
fe r a
SALES CUSTOMERS
ans
SALES 7 77111
n - t r
optimized physical reads
a no 473

) has eฺ
SQL>
ฺ c om Guid
e
16. Back in the terminal session connected to qr01cel01,
t nt the Exadata cell command-
ch delaunch
line interface (CellCLI).
yi nfo Stu
[celladmin@qr01cel01 e t h is
z~]$ cellcli
CellCLI: Release
o h i@ s e – Production...
11.2.3.2.1
u
s b irs e to
( ja ens
CellCLI>
i
S oh lic
a s bir
J

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 17
17. Use the tablespace number (TS#) and object number (DATAOBJ#) you gathered in step 15
to query the Exadata Smart Flash Cache. The output relates specifically to the
SALES.CUSTOMERS table.
CellCLI> list flashcachecontent where objectnumber=77111 -
> and tablespacenumber=7 and dbuniquename=DBM detail
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

cachedKeepSize: 0
cachedSize: 10362880
dbID: 2080757153
dbUniqueName: DBM
hitCount: 150
missCount: 146
objectNumber: 77111
tableSpaceNumber: 7
ble
fe r a
CellCLI>
t r a ns
Note that in step 15 the value for optimized physical reads is 473owhile
n n- the
a time allocated for
hitCount observed in this step is 150. Why is this so? In the remaining
s
this practice, query the Exadata Smart Flash Cache hitCount
) h a values ฺfor the other cells
e
d all the cells with
total iacross
(qr01cel02 and qr01cel03) and compare the hitCount
ฺ c oinmstep G uExplain
the optimized physical reads value observed
e c h ent 15. your observations.
18. Exit your terminal sessions.
i n fot Stud
e zy this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
a s bir
J

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 18
Practice 4-4: Storage Index
Overview
In this practice, you are introduced to the storage index capability of Exadata. You will execute a
query multiple times and examine statistics to measure the effect of storage index.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Tasks
1. Establish a terminal connection to qr01db01 as the oracle user.
2. Connect to your database with SQL*Plus. Log in as the sales user.
[oracle@qr01db01 ~]$ sqlplus sales/sales

SQL*Plus: Release 11.2.0.3.0 Production...

ble
SQL>
fe r a
3. Execute the following query (or execute the SQL script /home/oracle/labs/lab04-
ans
04-03.sql) and verify that the statistics are at or near zero values: n - t r
SQL> select a.name, b.value/1024/1024 MB
a no
2 from v$sysstat a, v$mystat b
) has eฺ
om Guid
3 where a.statistic# = b.statistic# and
4
ฺ c
(a.name in ('physical read total bytes',
5
t e ch dent
'physical write total bytes',
6
nfo Stu
'cell IO uncompressed bytes')
yi
7 z this
or a.name like 'cell phy%');
e
o h i@ use
NAME
b sei r s t o MB

( j a sread
----------------------------------------------------------------
n bytes
----------

h i
physical
l i c etotal 0

ir Socell physical IO interconnect bytes


physical write total bytes 0

b 0

Jas cell
cell
physical IO bytes saved during optimized file creation
physical IO bytes saved during optimized RMAN file restore
0
0
cell physical IO bytes eligible for predicate offload 0
cell physical IO bytes saved by storage index 0
cell physical IO bytes sent directly to DB node to balance CPU 0
cell physical IO interconnect bytes returned by smart scan 0
cell IO uncompressed bytes 0

10 rows selected.

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 19
4. Execute the following query (or execute the SQL script /home/oracle/labs/lab04-
04-04.sql).
SQL> select cust_gender,count(*) from mycustomers
2 where cust_income_level = 'C: 50,000 - 69,999'
3 group by cust_gender;
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

C COUNT(*)
- ----------
M 47924
F 27300

SQL>
5. Repeat the statistics query from step 3 (or execute the SQL script ble
fe r a
/home/oracle/labs/lab04-04-03.sql).
ans
SQL> select a.name, b.value/1024/1024 MB
n - t r
no
2 from v$sysstat a, v$mystat b
3 where a.statistic# = b.statistic# and a
4 (a.name in ('physical read total bytes',
) has eฺ
om Guid
5 'physical write total bytes',
ฺ c
ch dent
6 'cell IO uncompressed bytes')
7 or a.name like 'cell phy%');
t e
fo Stu
yi n
NAME
e z this MB

physical read s o hi@ bytes us


e
---------------------------------------------------------------- ----------

b i r totale bytes
total
t o 204.484375

j a s IOnsinterconnect bytes
physical write
cell (physical
0

hi physicalliceIO bytes saved during optimized file creation


1.07327271
ocell
S cell physical IO bytes saved during optimized RMAN file restore 0
ir
asb
0

J cell
cell
physical IO bytes eligible for predicate offload
physical IO bytes saved by storage index
204.4375
0
cell physical IO bytes sent directly to DB node to balance CPU 0
cell physical IO interconnect bytes returned by smart scan 1.02639771
cell IO uncompressed bytes 204.4375

10 rows selected.

SQL>
The statistics show that the query in step 4 was conducted using Smart Scan. Note,
however, that cell physical IO bytes saved by storage index is zero. This is
because storage indexes are memory structures which do not persist when the Exadata
cells are restarted. They are dynamically built when tables are referenced for the first time
after the cells restart. Now that the mycustomers table has been scanned as a result of
the query in step 4, all subsequent queries on the mycustomers table can benefit from
whatever storage indexes the Exadata cells automatically create.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 20
6. Reconnect to your database in order to reset the session-level statistics.
SQL> connect sales/sales
Connected.
SQL>
7. Repeat the statistics query from step 3 (or execute the SQL script
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

/home/oracle/labs/lab04-04-03.sql) and verify that the statistics are again at or


near zero values:
SQL> select a.name, b.value/1024/1024 MB
2 from v$sysstat a, v$mystat b
3 where a.statistic# = b.statistic# and
4 (a.name in ('physical read total bytes',
5 'physical write total bytes',
6 'cell IO uncompressed bytes')
7 or a.name like 'cell phy%');
ble
fe r a
NAME MB
ans
---------------------------------------------------------------- ----------
n - t r
physical read total bytes n 0 o
a
has eฺ
physical write total bytes 0
cell physical IO interconnect bytes
) 0

ฺ c om Guid
cell physical IO bytes saved during optimized file creation 0

t e ch dent
cell physical IO bytes saved during optimized RMAN file restore 0

nfo Stu
cell physical IO bytes eligible for predicate offload 0

yi
cell physical IO bytes saved by storage index
z this 0
e
cell physical IO bytes sent directly to DB node to balance CPU 0

o h i@ use
cell physical IO interconnect bytes returned by smart scan 0

b irs e to
cell IO uncompressed bytes
s
0

i ( jaselected.
c e ns
S oh
10 rows
li
ir
Jasb8.
SQL>
Re-execute the query from step 4 (or execute the SQL script
/home/oracle/labs/lab04-04-04.sql).
SQL> select cust_gender,count(*) from mycustomers
2 where cust_income_level = 'C: 50,000 - 69,999'
3 group by cust_gender;

C COUNT(*)
- ----------
M 47924
F 27300

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 21
9. Repeat the statistics query from step 3 (or execute the SQL script
/home/oracle/labs/lab04-04-03.sql).
SQL> select a.name, b.value/1024/1024 MB
2 from v$sysstat a, v$mystat b
3 where a.statistic# = b.statistic# and
4 (a.name in ('physical read total bytes',
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

5 'physical write total bytes',


6 'cell IO uncompressed bytes')
7 or a.name like 'cell phy%');

NAME MB
---------------------------------------------------------------- ----------
physical read total bytes 204.4375
physical write total bytes 0
ble
cell physical IO interconnect bytes
cell physical IO bytes saved during optimized file creation
1.00943756
0
fe r a
ans
cell physical IO bytes saved during optimized RMAN file restore
cell physical IO bytes eligible for predicate offload
n - t r 0
204.4375
cell physical IO bytes saved by storage index
a no 111.140625

has eฺ
cell physical IO bytes sent directly to DB node to balance CPU 0
)
om Guid
cell physical IO interconnect bytes returned by smart scan 1.00943756
cell IO uncompressed bytes
ฺ c
h ent
93.296875

t e c
10 rows selected.
y i nfo Stud
e z this
SQL>
o h i@ u s ein step 8 benefits from the storage index. Instead of
conducting more
s b irs 204eMB
This time you will see
than
that the query
toof I/O inside the cells, storage indexes were used to bypass
s In other words, approximately 93 MB of I/O was conducted
a MB eofnI/O.
more than(j111
instead
i
ohfewer c
li Queries that benefit from storage indexes can execute more quickly
of 204 MB.
ir S
using resources which allows other workloads to benefit from the unused I/O
b
Jas10. resources.
Exit your SQL*Plus session.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Key Capabilities of Exadata Database Machine


Chapter 4 - Page 22
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
Practices )for s
haLesson
m i d eฺ 5:
Exadataฺ co Database
G u Machine
h
Initial
e e n t
c Configuration
o t d
y i nf Chapter
S tu5
ez thi s
h i @ s e
o
irs e to u
s b
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 1
Practices for Lesson 5
Practices Overview
In this practice, you will be introduced to the Oracle Exadata Deployment Assistant
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 2
Practice 5-1: Using the Oracle Exadata Deployment Assistant
Overview
In this practice, you will be introduced to the Oracle Exadata Deployment Assistant. You will use
the assistant to generate a set of configuration files for an example Database Machine
implementation scenario.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Tasks
1. Establish a terminal session connected to qr01db01 using the oracle OS user. Ensure
that you specify the –X option for ssh.
$ ssh –X oracle@qr01db01
oracle@qr01db01 password: <oracle>
ble
[oracle@qr01db01 ~]$
fe r a
2. The Oracle Exadata Configuration Assistant is bundled in the patch containing the ans
OneCommand configuration utility for Database Machine. In your laboratory environment, n - t r
the Oracle Exadata Configuration Assistant is located under
a no
containing the Oracle Exadata Configuration Assistant. ) has eฺ
/home/oracle/labs/Exaconf on qr01db01. Change directory to the directory

ฺ c om Guid
[oracle@qr01db01 ~]$ cd labs/Exaconf
[oracle@qr01db01 Exaconf]$ tec
h ent
3. Start the Oracle Exadata Configuration y i o
nfAssistant.
S tud
[oracle@qr01db01i@
ez
Exaconf]$th
is
o h u s e ./exaconf.sh

s b irs e to
4. By using the
i ( c e nsthe
jainformation in the following table, populate the Oracle Exadata Configuration

S ohExamine the
Assistant
table.
pages.
li options and
Leave default values for fields that are not specified in the following
additional information presented on each page. Proceed
b i runtil you reach the Review and Edit Details page.
Jas Step Window/Page Description Choices or Values
a. Customer Details Customer Name: Example Industries
Application: Example Full Rack
b. Hardware Selection This is your deployment: X3-2 Full Rack HP
c. Networking Examine the IP address requirements summary.
d. Administration Network Starting IP Address for Pool: 10.7.7.101
Gateway: 10.7.7.1
e. Client Ethernet Network Starting IP Address for Pool: 172.16.1.101
Gateway: 172.16.1.1
f. InfiniBand Network Leave the default values.
g. Backup / Data Guard Leave unconfigured.
Ethernet Network

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 3
Step Window/Page Description Choices or Values
h. OS Configuration Domain Name: example.com
DNS Servers: 10.7.7.5
NTP Servers: 10.7.7.5
Check the option for “Separate Grid Infrastructure
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

owner from the Database Owner”


i. Home and Database Leave the default values.
j. Cell Alerting Leave unconfigured.
k. Oracle Configuration Leave unconfigured.
Manager
l. Auto Service Request Leave unconfigured.
m. Grid Control Agent Leave unconfigured. ble
fe r a
5. On the Review and Edit Details page, click Generate Configuration Data. Examine the
ans
configuration details on the page. When you have finished, click Next to proceed.
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 4
6. On the Complete page, click Create Files to create the configuration files for this
deployment scenario. Click OK in the confirmation dialog.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 5
7. At this point, you should see a dialog indicating that the configuration files were successfully
generated. The assistant will also open a new terminal window with the current directory set
to the location of the generated configuration files. You may use this terminal window to
examine the generated files. Run ls -R and confirm that you see a file listing similar to the
screen shot below.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 6
8. View the catalog.xml file (using the more command or vi if you prefer). This file, also
known as the Database Machine schematic file, is one of the main configuration files that
drives the Database Machine configuration process. Examine the file and confirm that the
details within it match your inputs to the deployment assistant. Take a few minutes to
examine the other configuration files as well.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 7
9. After you are satisfied, click OK in the dialog window to complete the Oracle Exadata
Deployment Assistant session.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
( ja youenhaves now used the Oracle Exadata Deployment Assistant to generate
i
Congratulations, c
oofh configuration
li files for a database Machine deployment scenario. In the final part of
aSset
b irthis practice you will use the deployment assistant to import your configuration information
Jas and make some changes.
10. By using your original terminal window, start the Oracle Exadata Configuration Assistant.
[oracle@qr01db01 Exaconf]$ ./exaconf.sh

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 8
11. On the Welcome page, click Import.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 9
12. Use the Import Configuration File dialog to select the catalog.xml file under
/tmp/OneCommand/Example Industries.Example Full Rack/dm01. After you
select the file, click Open to proceed.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 10
13. Click No in the Import generated data dialog box. This instructs the deployment assistant to
only import the inputs gathered during the original deployment assistant session, and not all
of the consequently generated settings. Finally, click OK in the dialog box indicating that the
file import is complete.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 11
14. Navigate to the Administration Network page. Along the way, confirm that the deployment
assistant pages contain details from your previously generated configuration.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 12
15. Imagine that you wish to implement a non-default host naming convention. Click Modify.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 13
16. In the Admin Network Format Masks dialog, modify the Compute Node Name to dm01db%%
and modify the Cell Node Name to dm01cel%%. Notice that as you make these changes,
the associated ILOM hosts names also change. Finally, click Save to accept the
modifications.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 14
17. Notice how the change alters the sample first host names displayed in the Administration
Network Page. Similar capabilities are also available on the Client Ethernet Network and
InfiniBand Network pages.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 15
18. Navigate to the Home and Database page.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 16
19. Modify the disk group redundancy setting to NORMAL for both disk groups. Notice the
resulting change in disk group sizing information.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 17
20. Check the option to reserve additional space in the RECO disk group for a full database
backup. Notice the resulting change in disk group sizing information.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 18
21. Navigate to the Review and Edit Details page and click Generate Configuration Data.
Notice how the host names have changed to reflect the changes you made in step 16.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 19
22. Generate an updated set of configuration files and examine the updated catalog.xml file
to verify the modified disk group settings.
Hint: See steps 6-8 earlier in this practice.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 20
23. After you are satisfied, click OK in the dialog box to complete the Oracle Exadata
Deployment Assistant session.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
24. i
Exit your (
terminal c e ns
ja sessions.
h li
ir So
b
Jas

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 21
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Exadata Database Machine Initial Configuration


Chapter 5 - Page 22
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
Practices )for s
haLesson
m i d eฺ 6:
Exadataฺ co Storage
G u Server
h
c den
Configuration
e t
o t
y i nf Chapter
S tu6
ez thi s
h i @ s e
o
irs e to u
s b
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 1
Practices for Lesson 6
Practices Overview
In these practices, you will perform a variety of Exadata configuration tasks, including cell
configuration and storage reconfiguration. You will also consume Exadata storage using ASM,
configure Exadata storage security, exercise the privileges associated with the different cell user
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

accounts and use the distributed command line utility (dcli).

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 2
Practice 6-1: Cell Configuration
Overview
In this practice you examine, set, and validate some Exadata cell parameters.

Tasks
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

1. Establish a terminal connection to the qr01cel01 Exadata cell as the celladmin user.
2. Launch the Exadata cell command-line interface (CellCLI).
[celladmin@qr01cel01 ~]$ cellcli
CellCLI: Release 11.2.3.2.1 – Production...

CellCLI>
3. Execute the following CellCLI command to examine the attributes of the cell.
ble
CellCLI> list cell detail
fe r a
name: qr01cel01
ans
bbuTempThreshold: 60
n - t r
bbuChargeThreshold: n 800 o
a
has eฺ
bmcType: absent
cellVersion:
) OSS_11.2.3.2.1_LINUX.X64_130109
cpuCount:
ฺ c om Guid 1
diagHistoryDays:
t e ch dent 7

nfo Stu
fanCount: 1/1
fanStatus: yi
z this normal
e
i@ use
flashCacheMode: WriteThrough
id:
o h 8ab50138-a667-4793-a976-c540dc1930c5

s b irs e to
interconnectCount: 3

i ( ja ens
interconnect1: eth1

S oh lic
iormBoost: 0.0

ir ipaddress1: 192.168.1.103/24

Jasb kernelVersion:
makeModel:
2.6.32-400.11.1.el5uek
Fake hardware
metricHistoryDays: 7
offloadEfficiency: 596.0
powerCount: 1/1
powerStatus: normal
releaseVersion: 11.2.3.2.1
releaseTrackingBug: 14522699
status: online
temperatureReading: 0.0
temperatureStatus: normal
upTime: 0 days, 1:55
cellsrvStatus: running
msStatus: running
rsStatus: running

CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 3
4. Configure the cell to send email alerts to a fictitious Exadata administrator.
CellCLI> alter cell smtpServer='my_mail.example.com', -
> smtpFromAddr='john.doe@example.com', -
> smtpFrom='John Doe', -
> smtpToAddr='jane.smith@example.com', -
> notificationPolicy='critical,warning,clear', -
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

> notificationMethod='mail'
Cell qr01cel01 successfully altered

CellCLI>
5. Re-examine the cell configuration to verify the changes you made in step 4.
CellCLI> list cell detail
name: qr01cel01
bbuTempThreshold: 60
ble
bbuChargeThreshold: 800
fe r a
bmcType: absent
ans
cellVersion: OSS_11.2.3.2.1_LINUX.X64_130109
n - t r
cpuCount:
n 1 o
a
has eฺ
diagHistoryDays: 7
fanCount: 1/1
)
fanStatus:
ฺ c om Guid normal

ch dent
flashCacheMode: WriteThrough
id: t e 8ab50138-a667-4793-a976-c540dc1930c5

i nfo Stu
interconnectCount:
y 3

e z this
interconnect1: eth1

h i@ use
iormBoost:
o
0.0

s b irs e to
ipaddress1: 192.168.1.103/24

i ( ja ens
kernelVersion: 2.6.32-400.11.1.el5uek

oh lic
makeModel: Fake hardware

ir S metricHistoryDays: 7

asb
notificationMethod: mail
J notificationPolicy:
offloadEfficiency:
critical,warning,clear
596.0
powerCount: 1/1
powerStatus: normal
releaseVersion: 11.2.3.2.1
releaseTrackingBug: 14522699
smtpFrom: "John Doe"
smtpFromAddr: john.doe@example.com
smtpServer: my_mail.example.com
smtpToAddr: jane.smith@example.com
status: online
temperatureReading: 0.0
temperatureStatus: normal
upTime: 0 days, 1:56
cellsrvStatus: running
msStatus: running
rsStatus: running
CellCLI>
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 4
6. Execute the following CellCLI command to validate the email attributes configured for the
cell.
Note: Executing this command attempts to send a test email to each of the configured
email addresses. The validation process only confirms the ability to successfully send a test
email using the specified configuration. The validation process does not confirm the
existence of the target email account, nor does it confirm successful receipt of the test
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

email message. In this case an error message is observed because the target email server
(my_mail.example.com) does not really exist.
CellCLI> alter cell validate mail

CELL-02578: An error was detected in the SMTP configuration:


CELL-05503: An error was detected during notification. The text
of the associated internal error is: Unknown SMTP host:
my_mail.example.com.
ble
The notification recipient is jane.smith@example.com.
fe r a
ans
Please verify your SMTP configuration.
n - t r
a no
CellCLI>
) h as ฺ
c o m uide check of the cell
7. Execute the following CellCLI command to perform a complete internal

chฺ dent G
configuration settings.
CellCLI> alter cell validatete configuration
Cell qr01cel01 successfully
yi nfo altered
S tu
ez thi s
h i @ s e
CellCLI> o u
irs CELLe tVALIDATE
o
a s b
Note that the ALTER
s CONFIGURATION command does not perform I/O
( j n
o hi to perform
tests against
ice such
the cell’s
l hard disks and flash modules. You must use the CALIBRATE

i r S
command tests. The CALIBRATE command can only be executed in a
b CellCLI session initiated by the root user.
Jas8. Exit your CellCLI session.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 5
Practice 6-2: Storage Reconfiguration
Overview
In this practice, you alter the Database Machine storage configuration. The approach used in
this practice allows the storage reconfiguration to occur while the system is running and
databases remain available. The procedure is based on one of the methods described in the
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

section entitled Resizing Storage Grid Disks in the Oracle Exadata Database Machine Owner’s
Guide.
In this practice you will reconfigure the RECO_QR01 ASM disk group. The aim is to resize the
disk group and underlying Exadata grid disks so that some of the available free space can be
used to create another disk group in the following practice.
Often storage is reconfigured to alter the balance of space allocated to the default DATA and
RECO disk groups. In such cases, both disk groups are reconfigured in parallel, with the space
freed from one disk group immediately consumed by the other disk group. See the Oracle
Exadata Database Machine Owner’s Guide for more information. ble
fe r a
Note: To complete this practice successfully, you must follow the instructions carefully and
ans
replicate the commands exactly. Failure to do so could result in unrecoverable damage to your
lab environment. Please take care. n - t r
a no
Tasks
) h as ฺ
1. Establish a terminal connection to qr01db01 as the grid
c o m user.uide
e c hฺ ent G
2. Using SQL*Plus, connect to ASM as an ASM administrator.
[grid@qr01db01 ~]$ sqlplusfo/t as sysasm
yi n S tud
e z this
SQL*Plus: Release
h i @ se Production...
11.2.0.3.0

b i rso e to u
SQL> as
i ( j
c e ns
3. Examine h the ASM
oSQL li disk groups in your environment using the following query (or execute
S
irapproach used in the practice, the amount of free space in theFor
the script /home/oracle/labs/lab06-02-03.sql). the reconfiguration
s b disk group must exceed the
Ja associated REQUIRED_MIRROR_FREE_MB value. Ensure this is the case for the
RECO_QR01 disk group.
SQL> select name, total_mb, free_mb, required_mirror_free_mb
2 from v$asm_diskgroup;

NAME TOTAL_MB FREE_MB REQUIRED_MIRROR_FREE_MB


------------------------------ ---------- ---------- -----------------------
DATA_QR01 21312 10596 7104
DBFS_DG 5760 4424 1920
RECO_QR01 28800 27604 9600

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 6
4. The following query shows a summary of the space utilization for the disks in each disk
group (use the SQL script /home/oracle/labs/lab06-02-04.sql if you prefer). For
each disk group the query shows the number of associated disks, the size of each disk and
the minimum and maximum amount of free space on the disks. Examine the output to
ensure that the disk group being reconfigured is reasonably well balanced. In particular
ensure that none of the disks are at or near capacity because this may cause problems with
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

later rebalancing operations. If required rebalance the disk group using the ALTER
DISKGROUP ... REBALANCE command prior to proceeding.
SQL> select dg.name, count(*), d.total_mb,
2 min(d.free_mb) MIN_FREE_MB, max(d.free_mb) MAX_FREE_MB
3 from v$asm_disk d, v$asm_diskgroup dg
4 where dg.group_number=d.group_number and d.mount_status='CACHED'
5 group by dg.name, d.total_mb;

NAME COUNT(*) TOTAL_MB MIN_FREE_MB MAX_FREE_MB


ble
------------------------------ ---------- ---------- ----------- -----------
fe r a
DBFS_DG 36 160 88 136
ans
DATA_QR01 36 592 268 344
n - t r
RECO_QR01 36 800 748
a no 788

SQL> ) has eฺ
5. Check that no ASM rebalance operations are currently ฺ c om activeG uid the cluster.
across
SQL> select * from gv$asm_operation; t e ch dent
yi nfo Stu
no rows selected e z this
o h i@ use
SQL>
s b irs e to
(
6. Drop thei disks c e ns with the cell qr01cel01 (use the SQL script
ja associated
S oh li
b i rrebalance is specified in order to maintain dataif you
/home/oracle/labs/lab06-02-06.sql prefer). Note that a disk group
redundancy.
Jas SQL> alter diskgroup reco_qr01
2 drop disks in failgroup qr01cel01
3 rebalance power 11;

Diskgroup altered.

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 7
7. Monitor the rebalance operation using the following query.
SQL> select * from gv$asm_operation;

INST_ID GROUP_NUMBER OPERA STAT POWER ACTUAL SOFAR EST_WORK


---------- ------------ ----- ---- ---------- ---------- ---------- ----------
EST_RATE EST_MINUTES ERROR_CODE
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

---------- ----------- --------------------------------------------


1 3 REBAL RUN 11 11 60 199
225 0

SQL>
8. Periodically repeat the query to monitor the rebalance operation. When the query returns no
results, the rebalance operation is completed. Do not proceed to the next step until the
rebalance operation completes. ble
fe r a
SQL> select * from gv$asm_operation;
ans
n - t r
no rows selected
a no
) has eฺ
om Guid
SQL>
9. Use the following query (or execute the SQL script ฺ c
h ent/home/oracle/labs/lab06-02-
09.sql) to confirm that the disks are droppedt e c(HEADER_STATUS=FORMER
d and
f o
in s S t u
MOUNT_STATUS=CLOSED).
z y
SQL> select path, free_mb,
i @ e t hi
e header_status, mount_status
2 from v$asm_disk
s h us
olike '%RECO_QR01%cel01';
i r t o
sb nse
3 where path

( j a
o hi
PATH
l ice
b i r S ------------------------------------------------------------------------------
Jas
FREE_MB HEADER_STATU MOUNT_S
---------- ------------ -------
o/192.168.1.103/RECO_QR01_CD_11_qr01cel01
0 FORMER CLOSED

o/192.168.1.103/RECO_QR01_CD_08_qr01cel01
0 FORMER CLOSED

o/192.168.1.103/RECO_QR01_CD_00_qr01cel01
0 FORMER CLOSED

o/192.168.1.103/RECO_QR01_CD_06_qr01cel01
0 FORMER CLOSED

o/192.168.1.103/RECO_QR01_CD_10_qr01cel01
0 FORMER CLOSED

o/192.168.1.103/RECO_QR01_CD_01_qr01cel01

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 8
0 FORMER CLOSED

o/192.168.1.103/RECO_QR01_CD_04_qr01cel01

PATH
------------------------------------------------------------------------------
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

FREE_MB HEADER_STATU MOUNT_S


---------- ------------ -------
0 FORMER CLOSED

o/192.168.1.103/RECO_QR01_CD_07_qr01cel01
0 FORMER CLOSED

o/192.168.1.103/RECO_QR01_CD_05_qr01cel01
0 FORMER CLOSED
ble
fe r a
o/192.168.1.103/RECO_QR01_CD_09_qr01cel01
ans
0 FORMER CLOSED
n - t r
a no
has eฺ
o/192.168.1.103/RECO_QR01_CD_03_qr01cel01
0 FORMER CLOSED
)
o/192.168.1.103/RECO_QR01_CD_02_qr01cel01ฺc
om Guid
t e ch dent
nfo Stu
0 FORMER CLOSED

y i
z this
12 rows selected. i@
e
o h u s e
s b irs e to
SQL>
i ( c ns connection to the qr01cel01 Exadata cell as the
jaseparateeterminal
oh user.
10. Establish
S
a li Maintain your ASM administrator SQL session as you will require this
birthroughout the rest of the practice.
celladmin

J a s
11. Launch the Exadata cell command-line interface (CellCLI).
[celladmin@qr01cel01 ~]$ cellcli
CellCLI: Release 11.2.3.2.1 - Production...

CellCLI>
12. Examine the grid disks on qr01cel01. Notice the ASMModeStatus for the dropped disks.
CellCLI> list griddisk attributes name, size, ASMModeStatus
DATA_QR01_CD_00_qr01cel01 592M ONLINE
DATA_QR01_CD_01_qr01cel01 592M ONLINE
DATA_QR01_CD_02_qr01cel01 592M ONLINE
DATA_QR01_CD_03_qr01cel01 592M ONLINE
DATA_QR01_CD_04_qr01cel01 592M ONLINE
DATA_QR01_CD_05_qr01cel01 592M ONLINE
DATA_QR01_CD_06_qr01cel01 592M ONLINE
DATA_QR01_CD_07_qr01cel01 592M ONLINE
DATA_QR01_CD_08_qr01cel01 592M ONLINE
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 9
DATA_QR01_CD_09_qr01cel01 592M ONLINE
DATA_QR01_CD_10_qr01cel01 592M ONLINE
DATA_QR01_CD_11_qr01cel01 592M ONLINE
DBFS_DG_CD_00_qr01cel01 160M ONLINE
DBFS_DG_CD_01_qr01cel01 160M ONLINE
DBFS_DG_CD_02_qr01cel01 160M ONLINE
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

DBFS_DG_CD_03_qr01cel01 160M ONLINE


DBFS_DG_CD_04_qr01cel01 160M ONLINE
DBFS_DG_CD_05_qr01cel01 160M ONLINE
DBFS_DG_CD_06_qr01cel01 160M ONLINE
DBFS_DG_CD_07_qr01cel01 160M ONLINE
DBFS_DG_CD_08_qr01cel01 160M ONLINE
DBFS_DG_CD_09_qr01cel01 160M ONLINE
DBFS_DG_CD_10_qr01cel01 160M ONLINE
DBFS_DG_CD_11_qr01cel01 160M ONLINE
ble
RECO_QR01_CD_00_qr01cel01 800M UNUSED
fe r a
RECO_QR01_CD_01_qr01cel01 800M UNUSED
ans
RECO_QR01_CD_02_qr01cel01 800M UNUSED
n - t r
RECO_QR01_CD_03_qr01cel01
a no
800M UNUSED

has eฺ
RECO_QR01_CD_04_qr01cel01 800M UNUSED
RECO_QR01_CD_05_qr01cel01
) 800M UNUSED
RECO_QR01_CD_06_qr01cel01
ฺ c om Guid 800M UNUSED
RECO_QR01_CD_07_qr01cel01
t e ch dent 800M UNUSED

nfo Stu
RECO_QR01_CD_08_qr01cel01 800M UNUSED

yi
RECO_QR01_CD_09_qr01cel01
z this
800M UNUSED
e
RECO_QR01_CD_10_qr01cel01 800M UNUSED

o h i@ use
RECO_QR01_CD_11_qr01cel01 800M UNUSED

CellCLI> sb
irs e to
( s
jadisks eonnqr01cel01
13. h i
Drop the grid li c previously associated with the RECO_QR01 disk group.
ir SoCellCLI> drop
b griddisk all prefix=reco_qr01

Jas GridDisk
GridDisk
RECO_QR01_CD_00_qr01cel01
RECO_QR01_CD_01_qr01cel01
successfully
successfully
dropped
dropped
GridDisk RECO_QR01_CD_02_qr01cel01 successfully dropped
GridDisk RECO_QR01_CD_03_qr01cel01 successfully dropped
GridDisk RECO_QR01_CD_04_qr01cel01 successfully dropped
GridDisk RECO_QR01_CD_05_qr01cel01 successfully dropped
GridDisk RECO_QR01_CD_06_qr01cel01 successfully dropped
GridDisk RECO_QR01_CD_07_qr01cel01 successfully dropped
GridDisk RECO_QR01_CD_08_qr01cel01 successfully dropped
GridDisk RECO_QR01_CD_09_qr01cel01 successfully dropped
GridDisk RECO_QR01_CD_10_qr01cel01 successfully dropped
GridDisk RECO_QR01_CD_11_qr01cel01 successfully dropped

CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 10
14. Create a new set of grid disks. Use the same grid disk prefix as before, but specify a size of
480 MB for each grid disk.
CellCLI> create griddisk all harddisk prefix=RECO_QR01, size=480M
GridDisk RECO_QR01_CD_00_qr01cel01 successfully created
GridDisk RECO_QR01_CD_01_qr01cel01 successfully created
GridDisk RECO_QR01_CD_02_qr01cel01 successfully created
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

GridDisk RECO_QR01_CD_03_qr01cel01 successfully created


GridDisk RECO_QR01_CD_04_qr01cel01 successfully created
GridDisk RECO_QR01_CD_05_qr01cel01 successfully created
GridDisk RECO_QR01_CD_06_qr01cel01 successfully created
GridDisk RECO_QR01_CD_07_qr01cel01 successfully created
GridDisk RECO_QR01_CD_08_qr01cel01 successfully created
GridDisk RECO_QR01_CD_09_qr01cel01 successfully created
GridDisk RECO_QR01_CD_10_qr01cel01 successfully created
GridDisk RECO_QR01_CD_11_qr01cel01 successfully created
ble
fe r a
CellCLI>
t r a ns
n
15. Re-examine the grid disk on qr01cel01. Notice the reconfigured grid disks
o -with the
RECO_QR01 prefix.
a n
h a s
CellCLI> list griddisk attributes name, size, ASMModeStatus
592M ) ONLINE de

DATA_QR01_CD_00_qr01cel01
m
co tONLINEu i
DATA_QR01_CD_01_qr01cel01
DATA_QR01_CD_02_qr01cel01 ec
h ฺ592M G
592M n ONLINE

n f o t t u de ONLINE
i
zy this S 592M ONLINE
DATA_QR01_CD_03_qr01cel01 592M

e
DATA_QR01_CD_04_qr01cel01
@ se
h i
DATA_QR01_CD_05_qr01cel01 592M ONLINE

b i rso e to u
DATA_QR01_CD_06_qr01cel01 592M ONLINE

( j a sDATA_QR01_CD_08_qr01cel01
n s
DATA_QR01_CD_07_qr01cel01 592M ONLINE

h i l i c e 592M ONLINE

S o DATA_QR01_CD_09_qr01cel01 592M ONLINE

a s bir DATA_QR01_CD_10_qr01cel01 592M ONLINE

J DATA_QR01_CD_11_qr01cel01
DBFS_DG_CD_00_qr01cel01
592M ONLINE
160M ONLINE
DBFS_DG_CD_01_qr01cel01 160M ONLINE
DBFS_DG_CD_02_qr01cel01 160M ONLINE
DBFS_DG_CD_03_qr01cel01 160M ONLINE
DBFS_DG_CD_04_qr01cel01 160M ONLINE
DBFS_DG_CD_05_qr01cel01 160M ONLINE
DBFS_DG_CD_06_qr01cel01 160M ONLINE
DBFS_DG_CD_07_qr01cel01 160M ONLINE
DBFS_DG_CD_08_qr01cel01 160M ONLINE
DBFS_DG_CD_09_qr01cel01 160M ONLINE
DBFS_DG_CD_10_qr01cel01 160M ONLINE
DBFS_DG_CD_11_qr01cel01 160M ONLINE
RECO_QR01_CD_00_qr01cel01 480M UNUSED
RECO_QR01_CD_01_qr01cel01 480M UNUSED
RECO_QR01_CD_02_qr01cel01 480M UNUSED
RECO_QR01_CD_03_qr01cel01 480M UNUSED
RECO_QR01_CD_04_qr01cel01 480M UNUSED

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 11
RECO_QR01_CD_05_qr01cel01 480M UNUSED
RECO_QR01_CD_06_qr01cel01 480M UNUSED
RECO_QR01_CD_07_qr01cel01 480M UNUSED
RECO_QR01_CD_08_qr01cel01 480M UNUSED
RECO_QR01_CD_09_qr01cel01 480M UNUSED
RECO_QR01_CD_10_qr01cel01 480M UNUSED
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

RECO_QR01_CD_11_qr01cel01 480M UNUSED

CellCLI>
16. Exit your CellCLI session but keep your terminal session open. You will require a terminal
session connected to qr01cel01 as the celladmin user later in the practice.
CellCLI> exit
quitting

ble
[celladmin@qr01cel01 ~]$
fe r a
17. Back in your ASM administrator SQL session, re-execute the following query (or execute ans
the SQL script /home/oracle/labs/lab06-02-09.sql). Notice that the reconfigured n - t r
grid disks are listed with HEADER_STATUS=CANDIDATE.
a no
h as ฺ
SQL> select path, free_mb, header_status, mount_status
)
2 from v$asm_disk
3 where path like '%RECO_QR01%cel01'; ฺco
m uide
e c h ent G
PATH
i n fot Stud
e zy this
------------------------------------------------------------------------------

h i
FREE_MB HEADER_STATU @ MOUNT_S
s e
o
irs e to
---------- ------------ u
-------
s b
ns CLOSED
o/192.168.1.103/RECO_QR01_CD_04_qr01cel01

i ( ja 0 CANDIDATE
e
oh li c
ir S o/192.168.1.103/RECO_QR01_CD_00_qr01cel01
Jasb 0 CANDIDATE CLOSED

o/192.168.1.103/RECO_QR01_CD_02_qr01cel01
0 CANDIDATE CLOSED

o/192.168.1.103/RECO_QR01_CD_09_qr01cel01
0 CANDIDATE CLOSED

o/192.168.1.103/RECO_QR01_CD_03_qr01cel01
0 CANDIDATE CLOSED

o/192.168.1.103/RECO_QR01_CD_05_qr01cel01
0 CANDIDATE CLOSED

o/192.168.1.103/RECO_QR01_CD_08_qr01cel01

PATH
------------------------------------------------------------------------------

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 12
FREE_MB HEADER_STATU MOUNT_S
---------- ------------ -------
0 CANDIDATE CLOSED

o/192.168.1.103/RECO_QR01_CD_11_qr01cel01
0 CANDIDATE CLOSED
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

o/192.168.1.103/RECO_QR01_CD_07_qr01cel01
0 CANDIDATE CLOSED

o/192.168.1.103/RECO_QR01_CD_06_qr01cel01
0 CANDIDATE CLOSED

o/192.168.1.103/RECO_QR01_CD_10_qr01cel01
0 CANDIDATE CLOSED
ble
fe r a
o/192.168.1.103/RECO_QR01_CD_01_qr01cel01
ans
0 CANDIDATE CLOSED
n - t r
a no
12 rows selected.
) has eฺ
ฺ c om Guid
SQL>
e h ent
cRECO_QR01
18. Add the reconfigured grid disks back intof o t
the
t u d disk group, and at the same time
drop the disks associated with the yi n S (use the SQL script
e z cell qr01cel02
h i s
@ se t
/home/oracle/labs/lab06-02-18.sql
h i if you prefer).

b i rso e to ureco_qr01 add disk


SQL> alter diskgroup

jas ens
2 'o/192.168.1.103/RECO_QR01_CD_00_qr01cel01',
3i ( 'o/192.168.1.103/RECO_QR01_CD_01_qr01cel01',
S lic
oh4 'o/192.168.1.103/RECO_QR01_CD_02_qr01cel01',
a s bir 5 'o/192.168.1.103/RECO_QR01_CD_03_qr01cel01',
J 6 'o/192.168.1.103/RECO_QR01_CD_04_qr01cel01',
7 'o/192.168.1.103/RECO_QR01_CD_05_qr01cel01',
8 'o/192.168.1.103/RECO_QR01_CD_06_qr01cel01',
9 'o/192.168.1.103/RECO_QR01_CD_07_qr01cel01',
10 'o/192.168.1.103/RECO_QR01_CD_08_qr01cel01',
11 'o/192.168.1.103/RECO_QR01_CD_09_qr01cel01',
12 'o/192.168.1.103/RECO_QR01_CD_10_qr01cel01',
13 'o/192.168.1.103/RECO_QR01_CD_11_qr01cel01'
14 drop disks in failgroup qr01cel02
15 rebalance power 11;

Diskgroup altered.

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 13
19. Monitor the rebalance operation by using the following query.
SQL> select * from gv$asm_operation;

INST_ID GROUP_NUMBER OPERA STAT POWER ACTUAL SOFAR EST_WORK


---------- ------------ ----- ---- ---------- ---------- ---------- ----------
EST_RATE EST_MINUTES ERROR_CODE
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

---------- ----------- --------------------------------------------


1 3 REBAL RUN 11 11 78 145
162 0

SQL>
20. Periodically repeat the query to monitor the rebalance operation. When the query returns no
results the rebalance operation is completed. Do not proceed to the next step until the
rebalance operation completes. ble
fe r a
SQL> select * from gv$asm_operation;
ans
n - t r
no rows selected
a no
SQL> ) has eฺ
21. Re-execute the query from step 4 (use the SQL script ฺ c om G uid
04.sql if you prefer). Now you can see the e ch dendisk t group in a partially
/home/oracle/labs/lab06-02-
t RECO_QR01

yi nfo associated
reconfigured state. At this point the storage
S tu with the RECO_QR01 disk group has
been reconfigured on qr01cel01 ez (12 disks)s
thi and the disks on qr01cel02 have been
dropped. i @
h count(*),e
i r
SQL> select dg.name, s o t o us d.total_mb,
j a sb nseMIN_FREE_MB, max(d.free_mb) MAX_FREE_MB
2 min(d.free_mb)
(
o h34i from l ce
idg.group_number=d.group_number
v$asm_disk d, v$asm_diskgroup dg

i r S 5 group by dg.name, d.total_mb;


where and d.mount_status='CACHED'
b
Jas
NAME COUNT(*) TOTAL_MB MIN_FREE_MB MAX_FREE_MB
------------------------------ ---------- ---------- ----------- -----------
DBFS_DG 36 160 80 136
DATA_QR01 36 592 268 344
RECO_QR01 12 800 756 768
RECO_QR01 12 480 436 444

SQL>

22. Establish a separate terminal connection to the qr01cel02 Exadata cell as the
celladmin user.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 14
23. Launch the Exadata cell command-line interface (CellCLI).
[celladmin@qr01cel02 ~]$ cellcli
CellCLI: Release 11.2.3.2.1 - Production...

CellCLI>
24. Examine the grid disks on qr01cel02. Notice again the ASMModeStatus for the dropped
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

disks.
Note: If the ASMModeStatus for the RECO_QR01 prefixed grid disks is ONLINE then you
have either dropped the wrong disks in step 18 or you are connected to the wrong cell (you
should now be connected to qr01cel02). In either case, do not proceed to the next step
until you have resolved the problem.
CellCLI> list griddisk attributes name, size, ASMModeStatus
DATA_QR01_CD_00_qr01cel02 592M ONLINE
DATA_QR01_CD_01_qr01cel02 592M ONLINE
ble
DATA_QR01_CD_02_qr01cel02 592M ONLINE
fe r a
DATA_QR01_CD_03_qr01cel02 592M ONLINE
ans
DATA_QR01_CD_04_qr01cel02 592M ONLINE
n - t r
DATA_QR01_CD_05_qr01cel02 592M ONLINE
n o
DATA_QR01_CD_06_qr01cel02 592M
a ONLINE
DATA_QR01_CD_07_qr01cel02
has eฺ
592M
)
ONLINE

om Guid
DATA_QR01_CD_08_qr01cel02 592M ONLINE
DATA_QR01_CD_09_qr01cel02
ฺ c 592M ONLINE
DATA_QR01_CD_10_qr01cel02
t e ch dent592M ONLINE

nfo Stu
DATA_QR01_CD_11_qr01cel02 592M ONLINE
yi
DBFS_DG_CD_00_qr01cel02
z this 160M ONLINE
e
DBFS_DG_CD_01_qr01cel02 160M ONLINE

o h i@ use
DBFS_DG_CD_02_qr01cel02 160M ONLINE

s b irs e to
DBFS_DG_CD_03_qr01cel02 160M ONLINE

i ( ja ens
DBFS_DG_CD_04_qr01cel02 160M ONLINE

lic
DBFS_DG_CD_05_qr01cel02 160M ONLINE

S oh DBFS_DG_CD_06_qr01cel02 160M ONLINE


ir
asb
DBFS_DG_CD_07_qr01cel02 160M ONLINE

J DBFS_DG_CD_08_qr01cel02
DBFS_DG_CD_09_qr01cel02
160M
160M
ONLINE
ONLINE
DBFS_DG_CD_10_qr01cel02 160M ONLINE
DBFS_DG_CD_11_qr01cel02 160M ONLINE
RECO_QR01_CD_00_qr01cel02 800M UNUSED
RECO_QR01_CD_01_qr01cel02 800M UNUSED
RECO_QR01_CD_02_qr01cel02 800M UNUSED
RECO_QR01_CD_03_qr01cel02 800M UNUSED
RECO_QR01_CD_04_qr01cel02 800M UNUSED
RECO_QR01_CD_05_qr01cel02 800M UNUSED
RECO_QR01_CD_06_qr01cel02 800M UNUSED
RECO_QR01_CD_07_qr01cel02 800M UNUSED
RECO_QR01_CD_08_qr01cel02 800M UNUSED
RECO_QR01_CD_09_qr01cel02 800M UNUSED
RECO_QR01_CD_10_qr01cel02 800M UNUSED
RECO_QR01_CD_11_qr01cel02 800M UNUSED

CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 15
25. Drop the grid disks on qr01cel02 previously associated with the RECO_QR01 disk group.
CellCLI> drop griddisk all prefix=reco_qr01
GridDisk RECO_QR01_CD_00_qr01cel02 successfully dropped
GridDisk RECO_QR01_CD_01_qr01cel02 successfully dropped
GridDisk RECO_QR01_CD_02_qr01cel02 successfully dropped
GridDisk RECO_QR01_CD_03_qr01cel02 successfully dropped
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

GridDisk RECO_QR01_CD_04_qr01cel02 successfully dropped


GridDisk RECO_QR01_CD_05_qr01cel02 successfully dropped
GridDisk RECO_QR01_CD_06_qr01cel02 successfully dropped
GridDisk RECO_QR01_CD_07_qr01cel02 successfully dropped
GridDisk RECO_QR01_CD_08_qr01cel02 successfully dropped
GridDisk RECO_QR01_CD_09_qr01cel02 successfully dropped
GridDisk RECO_QR01_CD_10_qr01cel02 successfully dropped
GridDisk RECO_QR01_CD_11_qr01cel02 successfully dropped

ble
CellCLI>
fe r a
26. Create a new set of grid disks. Use the same grid disk prefix as before, but specify a size of ans
480 MB for each grid disk. n - t r
n o
CellCLI>
a
create griddisk all harddisk prefix=RECO_QR01, size=480M
GridDisk
has eฺ
RECO_QR01_CD_00_qr01cel02 successfully created
)
om Guid
GridDisk RECO_QR01_CD_01_qr01cel02 successfully created
GridDisk ฺ c
RECO_QR01_CD_02_qr01cel02 successfully created
GridDisk
t e ch dent
RECO_QR01_CD_03_qr01cel02 successfully created
GridDisk
i nfo Stu
RECO_QR01_CD_04_qr01cel02 successfully created
y
GridDisk
e z this
RECO_QR01_CD_05_qr01cel02 successfully created
GridDisk
i@ use
RECO_QR01_CD_06_qr01cel02 successfully created
o h
GridDisk

s b irs e to
RECO_QR01_CD_07_qr01cel02 successfully created

ja ens
GridDisk RECO_QR01_CD_08_qr01cel02 successfully created

i (
GridDisk RECO_QR01_CD_09_qr01cel02 successfully created

S oh
GridDisk lic
RECO_QR01_CD_10_qr01cel02 successfully created
ir
asb
GridDisk RECO_QR01_CD_11_qr01cel02 successfully created

J CellCLI>
27. Exit your CellCLI session.
CellCLI> exit
quitting

[celladmin@qr01cel02 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 16
28. Back in your ASM administrator SQL session, add the reconfigured grid disks on
qr01cel02 back into the RECO_QR01 disk group, and at the same time drop the disks
associated with the cell qr01cel03 (use the SQL script /home/oracle/labs/lab06-
02-28.sql if you prefer).
SQL> alter diskgroup reco_qr01 add disk
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

2 'o/192.168.1.104/RECO_QR01_CD_00_qr01cel02',
3 'o/192.168.1.104/RECO_QR01_CD_01_qr01cel02',
4 'o/192.168.1.104/RECO_QR01_CD_02_qr01cel02',
5 'o/192.168.1.104/RECO_QR01_CD_03_qr01cel02',
6 'o/192.168.1.104/RECO_QR01_CD_04_qr01cel02',
7 'o/192.168.1.104/RECO_QR01_CD_05_qr01cel02',
8 'o/192.168.1.104/RECO_QR01_CD_06_qr01cel02',
9 'o/192.168.1.104/RECO_QR01_CD_07_qr01cel02',
ble
10 'o/192.168.1.104/RECO_QR01_CD_08_qr01cel02',
fe r a
11 'o/192.168.1.104/RECO_QR01_CD_09_qr01cel02',
ans
12 'o/192.168.1.104/RECO_QR01_CD_10_qr01cel02', n - t r
13 'o/192.168.1.104/RECO_QR01_CD_11_qr01cel02' a no
14 drop disks in failgroup qr01cel03
) has eฺ
15 rebalance power 11;
ฺ c om Guid
t e ch dent
Diskgroup altered.
yi nfo Stu
e z this
SQL>
o h i@ use
s b irs operation
29. Monitor the rebalance
e t o as before.
(
SQL> select
i c ns gv$asm_operation;
ja * efrom
S oh INST_IDliGROUP_NUMBER OPERA STAT
b i r ---------- ------------ ----- ---- ----------
POWER ACTUAL SOFAR EST_WORK

Jas
---------- ---------- ----------
EST_RATE EST_MINUTES ERROR_CODE
---------- ----------- --------------------------------------------
1 3 REBAL RUN 11 11 39 125
131 0

SQL>
30. Periodically repeat the query to monitor the rebalance operation. When the query returns no
results, the rebalance operation is completed. Do not proceed to the next step until the
rebalance operation completes.
SQL> select * from gv$asm_operation;

no rows selected

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 17
31. Re-execute the query from step 4 (use the SQL script /home/oracle/labs/lab06-02-
04.sql if you prefer). Now the storage associated with the RECO_QR01 disk group has
been reconfigured on two cells (24 disks on qr01cel01 and qr01cel02) and the disks on
the remaining cell (qr01cel03) have been dropped.
SQL> select dg.name, count(*), d.total_mb,
2 min(d.free_mb) MIN_FREE_MB, max(d.free_mb) MAX_FREE_MB
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

3 from v$asm_disk d, v$asm_diskgroup dg


4 where dg.group_number=d.group_number and d.mount_status='CACHED'
5 group by dg.name, d.total_mb;

NAME COUNT(*) TOTAL_MB MIN_FREE_MB MAX_FREE_MB


------------------------------ ---------- ---------- ----------- -----------
DBFS_DG 36 160 80 136
DATA_QR01 36 592 268 344
RECO_QR01 24 480 432 452
ble
fe r a
SQL>
t r a ns
32. Establish a separate terminal connection to the qr01cel03 Exadata cell as
n o n-the
sa
celladmin user.
33. Launch the Exadata cell command-line interface (CellCLI). ha
m ) i d eฺ
[celladmin@qr01cel03 ~]$ cellcli
h ฺ co t Gu
CellCLI: Release 11.2.3.2.1 – Production...
t e c den
i nf o tu
y s S
CellCLI>
@ ez thi
34. Examine the grid disks on
o h i s
qr01cel03.
u e Notice again the ASMModeStatus for the dropped
disks.
s b irs e to
Note: If theja
i ( c e nswrongfordisks
ASMModeStatus the RECO_QR01 prefixed grid disks is ONLINE then you
o h
have either l i
dropped the in step 28 or you are connected to the wrong cell (you
S
irshould now be connected to qr01cel03). In either case, do not proceed to the next step
s b until you have resolved the problem.
Ja CellCLI> list griddisk attributes name, size, ASMModeStatus
DATA_QR01_CD_00_qr01cel03 592M ONLINE
DATA_QR01_CD_01_qr01cel03 592M ONLINE
DATA_QR01_CD_02_qr01cel03 592M ONLINE
DATA_QR01_CD_03_qr01cel03 592M ONLINE
DATA_QR01_CD_04_qr01cel03 592M ONLINE
DATA_QR01_CD_05_qr01cel03 592M ONLINE
DATA_QR01_CD_06_qr01cel03 592M ONLINE
DATA_QR01_CD_07_qr01cel03 592M ONLINE
DATA_QR01_CD_08_qr01cel03 592M ONLINE
DATA_QR01_CD_09_qr01cel03 592M ONLINE
DATA_QR01_CD_10_qr01cel03 592M ONLINE
DATA_QR01_CD_11_qr01cel03 592M ONLINE
DBFS_DG_CD_00_qr01cel03 160M ONLINE
DBFS_DG_CD_01_qr01cel03 160M ONLINE
DBFS_DG_CD_02_qr01cel03 160M ONLINE
DBFS_DG_CD_03_qr01cel03 160M ONLINE

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 18
DBFS_DG_CD_04_qr01cel03 160M ONLINE
DBFS_DG_CD_05_qr01cel03 160M ONLINE
DBFS_DG_CD_06_qr01cel03 160M ONLINE
DBFS_DG_CD_07_qr01cel03 160M ONLINE
DBFS_DG_CD_08_qr01cel03 160M ONLINE
DBFS_DG_CD_09_qr01cel03 160M ONLINE
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

DBFS_DG_CD_10_qr01cel03 160M ONLINE


DBFS_DG_CD_11_qr01cel03 160M ONLINE
RECO_QR01_CD_00_qr01cel03 800M UNUSED
RECO_QR01_CD_01_qr01cel03 800M UNUSED
RECO_QR01_CD_02_qr01cel03 800M UNUSED
RECO_QR01_CD_03_qr01cel03 800M UNUSED
RECO_QR01_CD_04_qr01cel03 800M UNUSED
RECO_QR01_CD_05_qr01cel03 800M UNUSED
RECO_QR01_CD_06_qr01cel03 800M UNUSED
ble
RECO_QR01_CD_07_qr01cel03 800M UNUSED
fe r a
RECO_QR01_CD_08_qr01cel03 800M UNUSED
ans
RECO_QR01_CD_09_qr01cel03 800M UNUSED
n - t r
RECO_QR01_CD_10_qr01cel03
a no 800M UNUSED

as ฺ
RECO_QR01_CD_11_qr01cel03 800M UNUSED

) h
CellCLI>
c o m uide
hฺ enwith
35. Drop the grid disks on qr01cel03 previouslycassociated
e t Gthe RECO_QR01 disk group.
n fot Stud
CellCLI> drop griddisk all prefix=reco_qr01
i
z y
GridDisk RECO_QR01_CD_00_qr01cel03
e i s
successfully dropped
t h
i@ use successfully dropped
GridDisk RECO_QR01_CD_01_qr01cel03 successfully dropped
o h
irs e to
GridDisk RECO_QR01_CD_02_qr01cel03

a s b
GridDisk RECO_QR01_CD_03_qr01cel03
s
successfully dropped

h i (j RECO_QR01_CD_04_qr01cel03
GridDisk
i c en successfully dropped

S l
oGridDisk RECO_QR01_CD_06_qr01cel03 successfully
GridDisk RECO_QR01_CD_05_qr01cel03 dropped

s b ir GridDisk RECO_QR01_CD_07_qr01cel03 successfully dropped


successfully

Ja GridDisk RECO_QR01_CD_08_qr01cel03
dropped
successfully dropped
GridDisk RECO_QR01_CD_09_qr01cel03 successfully dropped
GridDisk RECO_QR01_CD_10_qr01cel03 successfully dropped
GridDisk RECO_QR01_CD_11_qr01cel03 successfully dropped

CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 19
36. Create a new set of grid disks. Use the same grid disk prefix as before, but specify a size of
480 MB for each grid disk.
CellCLI> create griddisk all harddisk prefix=RECO_QR01, size=480M
GridDisk RECO_QR01_CD_00_qr01cel03 successfully created
GridDisk RECO_QR01_CD_01_qr01cel03 successfully created
GridDisk RECO_QR01_CD_02_qr01cel03 successfully created
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

GridDisk RECO_QR01_CD_03_qr01cel03 successfully created


GridDisk RECO_QR01_CD_04_qr01cel03 successfully created
GridDisk RECO_QR01_CD_05_qr01cel03 successfully created
GridDisk RECO_QR01_CD_06_qr01cel03 successfully created
GridDisk RECO_QR01_CD_07_qr01cel03 successfully created
GridDisk RECO_QR01_CD_08_qr01cel03 successfully created
GridDisk RECO_QR01_CD_09_qr01cel03 successfully created
GridDisk RECO_QR01_CD_10_qr01cel03 successfully created
GridDisk RECO_QR01_CD_11_qr01cel03 successfully created
ble
fe r a
CellCLI>
ans
37. Exit your CellCLI session. n - t r
CellCLI> exit a no
quitting
) has eฺ
c o m uid
[celladmin@qr01cel03 ~]$
e c h ent G

fot S
38. Back in your ASM administrator SQL session,
i n t
add d reconfigured grid disks on
uthe
y
z disk
qr01cel03 back into the RECO_QR01
e isgroup (use the SQL script
i @ se
/home/oracle/labs/lab06-02-38.sql t h if you prefer).
h
so to ureco_qr01 add disk
SQL> alter rdiskgroup
b i
s nse
j a
2 'o/192.168.1.105/RECO_QR01_CD_00_qr01cel03',
(
o l ice
h3i 'o/192.168.1.105/RECO_QR01_CD_01_qr01cel03',
b i r S 4 'o/192.168.1.105/RECO_QR01_CD_02_qr01cel03',
Jas 5 'o/192.168.1.105/RECO_QR01_CD_03_qr01cel03',
6 'o/192.168.1.105/RECO_QR01_CD_04_qr01cel03',
7 'o/192.168.1.105/RECO_QR01_CD_05_qr01cel03',
8 'o/192.168.1.105/RECO_QR01_CD_06_qr01cel03',
9 'o/192.168.1.105/RECO_QR01_CD_07_qr01cel03',
10 'o/192.168.1.105/RECO_QR01_CD_08_qr01cel03',
11 'o/192.168.1.105/RECO_QR01_CD_09_qr01cel03',
12 'o/192.168.1.105/RECO_QR01_CD_10_qr01cel03',
13 'o/192.168.1.105/RECO_QR01_CD_11_qr01cel03'
14 rebalance power 11;

Diskgroup altered.

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 20
39. Monitor the rebalance operation as before.
SQL> select * from gv$asm_operation;

INST_ID GROUP_NUMBER OPERA STAT POWER ACTUAL SOFAR EST_WORK


---------- ------------ ----- ---- ---------- ---------- ---------- ----------
EST_RATE EST_MINUTES ERROR_CODE
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

---------- ----------- --------------------------------------------


1 3 REBAL RUN 11 11 70 176
152 0

SQL>
40. Periodically repeat the query to monitor the rebalance operation. When the query returns no
results the rebalance operation is completed. Do not proceed to the next step until the
rebalance operation completes. ble
fe r a
SQL> select * from gv$asm_operation;
ans
n - t r
no rows selected
a no
) has eฺ
com uid
SQL>

41. Re-execute the query from step 4 (use the SQLhscript
c with G
t RECO_QR01 disk group has
/home/oracle/labs/lab06-02-
n
t e
04.sql if you prefer). Now the storage associated ethe
been reconfigured on all three cells. inf
y
o
S tud
e
SQL> select dg.name, count(*),
t h is
z d.total_mb,
2 min(d.free_mb)
o h i@ u s e max(d.free_mb) MAX_FREE_MB
MIN_FREE_MB,

b irs e d,tov$asm_diskgroup dg
3 from v$asm_disk
s
( j
4 wherea n s
dg.group_number=d.group_number
e
and d.mount_status='CACHED'

S oh5i grouplibyc dg.name, d.total_mb;


a s bir NAME COUNT(*) TOTAL_MB MIN_FREE_MB MAX_FREE_MB
J ------------------------------ ---------- ---------- ----------- -----------
DBFS_DG 36 160 80 136
DATA_QR01 36 592 268 344
RECO_QR01 36 480 432 460

SQL>

In the final part of this practice, the free space created by reconfiguring the RECO_QR01 disk
group will be provisioned into another set of grid disks.
42. Launch the Exadata cell command-line interface (CellCLI) on qr01cel01.
[celladmin@qr01cel01 ~]$ cellcli
CellCLI: Release 11.2.3.2.1 – Production...

CellCLI>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 21
43. Use the following CellCLI command to show the free space on each cell disk.
CellCLI> list celldisk attributes name, freeSpace where freeSpace != 0
CD_00_qr01cel01 320M
CD_01_qr01cel01 320M
CD_02_qr01cel01 320M
CD_03_qr01cel01 320M
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

CD_04_qr01cel01 320M
CD_05_qr01cel01 320M
CD_06_qr01cel01 320M
CD_07_qr01cel01 320M
CD_08_qr01cel01 320M
CD_09_qr01cel01 320M
CD_10_qr01cel01 320M
CD_11_qr01cel01 320M
ble
CellCLI>
fe r a
ans
44. Create a set of grid disks which consume all of the available free space. Specify
n - t r
prefix=DATA2_QR01.
n o
a
has eฺ
CellCLI> create griddisk all harddisk prefix=DATA2_QR01
GridDisk DATA2_QR01_CD_00_qr01cel01 successfully created
)
GridDisk
om Guid
DATA2_QR01_CD_01_qr01cel01 successfully created
ฺ c
ch dent
GridDisk DATA2_QR01_CD_02_qr01cel01 successfully created
t e
nfo Stu
GridDisk DATA2_QR01_CD_03_qr01cel01 successfully created
GridDisk
yi
DATA2_QR01_CD_04_qr01cel01 successfully created
z this
GridDisk e
DATA2_QR01_CD_05_qr01cel01 successfully created
GridDisk
h i@ use
DATA2_QR01_CD_06_qr01cel01 successfully created
o
GridDisk
s b irs e to
DATA2_QR01_CD_07_qr01cel01 successfully created

i ( ja ens
GridDisk DATA2_QR01_CD_08_qr01cel01 successfully created

S oh
GridDisk
GridDisk lic
DATA2_QR01_CD_09_qr01cel01 successfully created
DATA2_QR01_CD_10_qr01cel01 successfully created
ir
asb
GridDisk DATA2_QR01_CD_11_qr01cel01 successfully created
J
CellCLI>
45. Examine the newly created grid disks. Note that they are now ready to be consumed in an
ASM disk group.
CellCLI> list griddisk attributes name, size, ASMModeStatus -
> where name like 'DATA2.*'
DATA2_QR01_CD_00_qr01cel01 320M UNUSED
DATA2_QR01_CD_01_qr01cel01 320M UNUSED
DATA2_QR01_CD_02_qr01cel01 320M UNUSED
DATA2_QR01_CD_03_qr01cel01 320M UNUSED
DATA2_QR01_CD_04_qr01cel01 320M UNUSED
DATA2_QR01_CD_05_qr01cel01 320M UNUSED
DATA2_QR01_CD_06_qr01cel01 320M UNUSED
DATA2_QR01_CD_07_qr01cel01 320M UNUSED
DATA2_QR01_CD_08_qr01cel01 320M UNUSED
DATA2_QR01_CD_09_qr01cel01 320M UNUSED
DATA2_QR01_CD_10_qr01cel01 320M UNUSED

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 22
DATA2_QR01_CD_11_qr01cel01 320M UNUSED

CellCLI>
46. Exit your CellCLI session but keep your terminal session open.
CellCLI> exit
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

quitting

[celladmin@qr01cel01 ~]$
47. In step 44, a set of grid disks was created on qr01cel01. Use the following command to
create similar grid disks on qr01cel02 and qr01cel03.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel02,qr01cel03 cellcli -e \
> create griddisk all harddisk prefix=DATA2_QR01
qr01cel02: GridDisk DATA2_QR01_CD_00_qr01cel02 successfully created
ble
qr01cel02: GridDisk DATA2_QR01_CD_01_qr01cel02 successfully created
fe r a
qr01cel02: GridDisk DATA2_QR01_CD_02_qr01cel02 successfully created
ans
qr01cel02: GridDisk DATA2_QR01_CD_03_qr01cel02 successfully created
n - t r
qr01cel02: GridDisk DATA2_QR01_CD_04_qr01cel02 successfully created
n o
a
qr01cel02: GridDisk DATA2_QR01_CD_05_qr01cel02 successfully created

has eฺ
qr01cel02: GridDisk DATA2_QR01_CD_06_qr01cel02 successfully created
)
om Guid
qr01cel02: GridDisk DATA2_QR01_CD_07_qr01cel02 successfully created
ฺ c
ch dent
qr01cel02: GridDisk DATA2_QR01_CD_08_qr01cel02 successfully created

t e
qr01cel02: GridDisk DATA2_QR01_CD_09_qr01cel02 successfully created

i nfo Stu
qr01cel02: GridDisk DATA2_QR01_CD_10_qr01cel02 successfully created
y
e z this
qr01cel02: GridDisk DATA2_QR01_CD_11_qr01cel02 successfully created

h i@ use
qr01cel03: GridDisk DATA2_QR01_CD_00_qr01cel03 successfully created
o
s b irs e to
qr01cel03: GridDisk DATA2_QR01_CD_01_qr01cel03 successfully created

ja ens
qr01cel03: GridDisk DATA2_QR01_CD_02_qr01cel03 successfully created

i (
lic
qr01cel03: GridDisk DATA2_QR01_CD_03_qr01cel03 successfully created

S oh
qr01cel03: GridDisk DATA2_QR01_CD_04_qr01cel03 successfully created
ir
asb
qr01cel03: GridDisk DATA2_QR01_CD_05_qr01cel03 successfully created

J qr01cel03: GridDisk DATA2_QR01_CD_06_qr01cel03 successfully created


qr01cel03: GridDisk DATA2_QR01_CD_07_qr01cel03 successfully created
qr01cel03: GridDisk DATA2_QR01_CD_08_qr01cel03 successfully created
qr01cel03: GridDisk DATA2_QR01_CD_09_qr01cel03 successfully created
qr01cel03: GridDisk DATA2_QR01_CD_10_qr01cel03 successfully created
qr01cel03: GridDisk DATA2_QR01_CD_11_qr01cel03 successfully created
[celladmin@qr01cel01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 23
48. Use the following command to verify the existence of the newly created grid disks on all
three cells.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 "cellcli -e \
> list griddisk attributes name, size, ASMModeStatus \
> where name like \'DATA2.*\'"
qr01cel01: DATA2_QR01_CD_00_qr01cel01 320M UNUSED
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

qr01cel01: DATA2_QR01_CD_01_qr01cel01 320M UNUSED


qr01cel01: DATA2_QR01_CD_02_qr01cel01 320M UNUSED
qr01cel01: DATA2_QR01_CD_03_qr01cel01 320M UNUSED
qr01cel01: DATA2_QR01_CD_04_qr01cel01 320M UNUSED
qr01cel01: DATA2_QR01_CD_05_qr01cel01 320M UNUSED
qr01cel01: DATA2_QR01_CD_06_qr01cel01 320M UNUSED
qr01cel01: DATA2_QR01_CD_07_qr01cel01 320M UNUSED
qr01cel01: DATA2_QR01_CD_08_qr01cel01 320M UNUSED
qr01cel01: DATA2_QR01_CD_09_qr01cel01 320M UNUSED ble
qr01cel01: DATA2_QR01_CD_10_qr01cel01 320M UNUSED
fe r a
qr01cel01: DATA2_QR01_CD_11_qr01cel01 320M UNUSED
ans
qr01cel02: DATA2_QR01_CD_00_qr01cel02 320M UNUSED
n - t r
qr01cel02: DATA2_QR01_CD_01_qr01cel02 320M n
UNUSED o
a
has eฺ
qr01cel02: DATA2_QR01_CD_02_qr01cel02 320M UNUSED
)
om Guid
qr01cel02: DATA2_QR01_CD_03_qr01cel02 320M UNUSED


qr01cel02: DATA2_QR01_CD_04_qr01cel02
c 320M UNUSED

t e ch dent
qr01cel02: DATA2_QR01_CD_05_qr01cel02 320M UNUSED

nfo Stu
qr01cel02: DATA2_QR01_CD_06_qr01cel02 320M UNUSED
yi
z this
qr01cel02: DATA2_QR01_CD_07_qr01cel02 320M UNUSED
e
o h i@ use
qr01cel02: DATA2_QR01_CD_08_qr01cel02
qr01cel02: DATA2_QR01_CD_09_qr01cel02
320M
320M
UNUSED
UNUSED

s b irs e to
qr01cel02: DATA2_QR01_CD_10_qr01cel02 320M UNUSED

i ( ja ens
qr01cel02: DATA2_QR01_CD_11_qr01cel02 320M UNUSED

S oh lic
qr01cel03: DATA2_QR01_CD_00_qr01cel03 320M UNUSED

ir qr01cel03: DATA2_QR01_CD_01_qr01cel03 320M UNUSED

Jasb qr01cel03: DATA2_QR01_CD_02_qr01cel03


qr01cel03: DATA2_QR01_CD_03_qr01cel03
320M
320M
UNUSED
UNUSED
qr01cel03: DATA2_QR01_CD_04_qr01cel03 320M UNUSED
qr01cel03: DATA2_QR01_CD_05_qr01cel03 320M UNUSED
qr01cel03: DATA2_QR01_CD_06_qr01cel03 320M UNUSED
qr01cel03: DATA2_QR01_CD_07_qr01cel03 320M UNUSED
qr01cel03: DATA2_QR01_CD_08_qr01cel03 320M UNUSED
qr01cel03: DATA2_QR01_CD_09_qr01cel03 320M UNUSED
qr01cel03: DATA2_QR01_CD_10_qr01cel03 320M UNUSED
qr01cel03: DATA2_QR01_CD_11_qr01cel03 320M UNUSED
[celladmin@qr01cel01 ~]$
49. Exit your CellCLI and SQL*Plus sessions.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 24
Practice 6-3: Consuming Grid Disks by Using ASM
Overview
In this practice, you consume some newly created Exadata grid disks using ASM.

Assumptions
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Before beginning this practice you must complete Practice 6-2. Your ability to complete this
practice depends on the existence of the grid disks that are created in practice 6-2.

Tasks
1. Establish a terminal connection to qr01db01 as the grid user.
2. By using SQL*Plus, connect to ASM as an ASM administrator.
[grid@qr01db01 ~]$ sqlplus / as sysasm
ble
fe r a
SQL*Plus: Release 11.2.0.3.0 Production...
ans
n - t r
SQL>
a no
The ASM environment you are using is clustered across the entire Database
h a s Machine. All the
grid disks defined across all the Exadata cells are visible inside )gridASM. In theฺ
dinedifferent
previous practice
you configured some grid disks. Now you will consume those o m
c tG u i
disks ways using
ASM. h ฺ
c den
3. Execute the following query (or executefo teSQL
the u string.
tscript /home/oracle/labs/lab06-
03-03.sql). Note the capitalization y i ninside theSlike The output shows a series of
e z h i s
t This indicates that the disks do not belong to
h @
disks having HEADER_STATUS=CANDIDATE.
i s e
an ASM disk group and
i r s o to u consumed by ASM. Examine the PATH output. Can
are ready to be
you determine the
a s b se cell that contains each grid disk?
Exadata
j
SQL> (select
i enlike
o h l i c
name, header_status, path from v$asm_disk

ir S 2 where path 'o/%/DATA2%'

asb
3 and header_status='CANDIDATE';

J NAME HEADER_STATU
------------------------------ ------------
PATH
------------------------------------------------------------------------------
CANDIDATE
o/192.168.1.105/DATA2_QR01_CD_07_qr01cel03

CANDIDATE
o/192.168.1.103/DATA2_QR01_CD_07_qr01cel01

CANDIDATE
o/192.168.1.105/DATA2_QR01_CD_02_qr01cel03

CANDIDATE
o/192.168.1.105/DATA2_QR01_CD_00_qr01cel03

CANDIDATE

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 25
o/192.168.1.105/DATA2_QR01_CD_08_qr01cel03

CANDIDATE
o/192.168.1.105/DATA2_QR01_CD_06_qr01cel03

CANDIDATE
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

NAME HEADER_STATU
------------------------------ ------------
PATH
------------------------------------------------------------------------------
o/192.168.1.105/DATA2_QR01_CD_10_qr01cel03

CANDIDATE
o/192.168.1.105/DATA2_QR01_CD_09_qr01cel03
ble
fe r a
CANDIDATE
ans
o/192.168.1.105/DATA2_QR01_CD_03_qr01cel03
n - t r
n o
a
has eฺ
CANDIDATE
o/192.168.1.105/DATA2_QR01_CD_05_qr01cel03
)
CANDIDATE ฺc
om Guid
t e ch dent
nfo Stu
o/192.168.1.105/DATA2_QR01_CD_01_qr01cel03

yi
e t h is
z CANDIDATE
h i@ use
o/192.168.1.105/DATA2_QR01_CD_04_qr01cel03
o
s b irs e to
( j a e n s CANDIDATE

ohi lic
o/192.168.1.104/DATA2_QR01_CD_09_qr01cel02

ir S NAME
asb
HEADER_STATU
J ------------------------------ ------------
PATH
------------------------------------------------------------------------------

CANDIDATE
o/192.168.1.104/DATA2_QR01_CD_06_qr01cel02

CANDIDATE
o/192.168.1.104/DATA2_QR01_CD_07_qr01cel02

CANDIDATE
o/192.168.1.104/DATA2_QR01_CD_11_qr01cel02

CANDIDATE
o/192.168.1.104/DATA2_QR01_CD_08_qr01cel02

CANDIDATE
o/192.168.1.104/DATA2_QR01_CD_01_qr01cel02
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 26
CANDIDATE
o/192.168.1.104/DATA2_QR01_CD_10_qr01cel02

NAME HEADER_STATU
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

------------------------------ ------------
PATH
------------------------------------------------------------------------------
CANDIDATE
o/192.168.1.104/DATA2_QR01_CD_03_qr01cel02

CANDIDATE
o/192.168.1.104/DATA2_QR01_CD_00_qr01cel02

ble
CANDIDATE
fe r a
o/192.168.1.104/DATA2_QR01_CD_04_qr01cel02
ans
n - t r
CANDIDATE
n o
a
as ฺ
o/192.168.1.104/DATA2_QR01_CD_02_qr01cel02

) h
CANDIDATE
c o m uide
e c hฺ ent G
o/192.168.1.104/DATA2_QR01_CD_05_qr01cel02

i n fot Stud
zy this
CANDIDATE
e
o/192.168.1.103/DATA2_QR01_CD_03_qr01cel01

o h i@ use
b i rs e to CANDIDATE

( j a s ns
hi
NAME
lice
o------------------------------
S PATH
HEADER_STATU
------------
ir
Jasb ------------------------------------------------------------------------------
o/192.168.1.103/DATA2_QR01_CD_09_qr01cel01

CANDIDATE
o/192.168.1.103/DATA2_QR01_CD_08_qr01cel01

CANDIDATE
o/192.168.1.103/DATA2_QR01_CD_05_qr01cel01

CANDIDATE
o/192.168.1.103/DATA2_QR01_CD_06_qr01cel01

CANDIDATE
o/192.168.1.103/DATA2_QR01_CD_11_qr01cel01

CANDIDATE
o/192.168.1.103/DATA2_QR01_CD_04_qr01cel01

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 27
CANDIDATE
o/192.168.1.103/DATA2_QR01_CD_02_qr01cel01

NAME HEADER_STATU
------------------------------ ------------
PATH
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

------------------------------------------------------------------------------

CANDIDATE
o/192.168.1.103/DATA2_QR01_CD_10_qr01cel01

CANDIDATE
o/192.168.1.103/DATA2_QR01_CD_00_qr01cel01

CANDIDATE
ble
o/192.168.1.103/DATA2_QR01_CD_01_qr01cel01
fe r a
ans
CANDIDATE
n - t r
o/192.168.1.105/DATA2_QR01_CD_11_qr01cel03
a no
) h as ฺ
36 rows selected.
c o m uide
e c hฺ ent G
SQL>
f o t tud
i n Sways. You can add disks to an existing
Exadata grid disks are consumed inside
ASM disk group or you can create e zyASM
a new t
ASM h
in two
i s
disk group based on Exadata grid disks.
h i @ e
4. Select one of the disks
i r s usoutput to step 3 and add it to the existing
olistedtino the RECO disk

a s b se
group (use the SQL script /home/oracle/labs/lab06-03-04.sql if you prefer). Use
the PATH (ofj your selected
h i l i c en disk to identify it in the ADD DISK clause.
i r SoSQL> alter diskgroup reco_qr01
b
Jas
2 add disk 'o/192.168.1.103/DATA2_QR01_CD_00_qr01cel01'
3 rebalance power 0;

Diskgroup altered.

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 28
5. Verify that the disk is added to your disk group using the following query (or execute the
SQL script /home/oracle/labs/lab06-03-05.sql).
SQL> select dg.name GNAME, d.name DNAME, d.header_status, d.path
2 from v$asm_disk d left outer join
3 (select * from v$asm_diskgroup where group_number != 0) dg
4 on d.group_number = dg.group_number
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

5 where dg.name is not null and d.path like 'o/%/DATA2%';

GNAME DNAME HEADER_STATU


------------------------------ ------------------------------ ------------
PATH
------------------------------------------------------------------------------
RECO_QR01 DATA2_QR01_CD_00_QR01CEL01 MEMBER
o/192.168.1.103/DATA2_QR01_CD_00_qr01cel01
ble
fe r a
SQL>
t r a ns
6. Remove the recently added disk from your disk group using the following
n o n- command
SQL
(or execute the SQL script /home/oracle/labs/lab06-03-06.sql).
s a Note the use of
) ha idIt emay
WAIT to ensure that the statement does not return until the disk is completely removed from
the disk group and the associated rebalance operationm completes.
ฺ take a few
minutes for the command to complete.
h ฺ co t Gu
t e c den
SQL> alter diskgroup reco_qr01
i nf o tu
y
2 drop disk DATA2_QR01_CD_00_QR01CEL01
s S
3 rebalance power
@ ez11 wait;
thi
o h i us e
i r s t o
Diskgroup
( j a sb altered.
n se
hi lic e
S oSQL>
a s birCreate a new ASM disk group consuming all the grid disks created in the previous practice
7.
J (use the SQL script /home/oracle/labs/lab06-03-07.sql if you prefer).
SQL> create diskgroup data2_qr01 normal redundancy
2 disk 'o/*/DATA2_QR01*'
3 attribute 'compatible.rdbms' = '11.2.0.0.0',
4 'compatible.asm' = '11.2.0.0.0',
5 'cell.smart_scan_capable' = 'TRUE',
6 'au_size' = '4M';

Diskgroup created.

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 29
The newly created disk group can be used to house Oracle data files in the same way as
an ASM disk group based on any other storage. To complement the recommended
AU_SIZE setting of 4 MB, you should set the initial extent size to 8 MB for large segments.
The recommended approaches are discussed in the lesson entitled Optimizing Database
Performance with Exadata.
8. Examine your newly created disk group using the following query (or execute the SQL
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

script /home/oracle/labs/lab06-03-08.sql). Note how the grid disks from each


different Exadata cell are automatically grouped into separate failure groups.
SQL> select d.path, dg.name GNAME, d.failgroup, d.state
2 from v$asm_disk d, v$asm_diskgroup dg
3 where d.group_number = dg.group_number
4 and dg.name = 'DATA2_QR01';

PATH
------------------------------------------------------------------------------
ble
GNAME FAILGROUP STATE
fe r a
------------------------------ ------------------------------ --------
ans
o/192.168.1.105/DATA2_QR01_CD_07_qr01cel03
n - t r
DATA2_QR01 QR01CEL03 n NORMAL o
a
o/192.168.1.105/DATA2_QR01_CD_11_qr01cel03
) has eฺ
DATA2_QR01 QR01CEL03
ฺ c om Guid NORMAL

t e ch dent
i fo Stu
o/192.168.1.105/DATA2_QR01_CD_02_qr01cel03
nQR01CEL03
DATA2_QR01
e y
z this NORMAL

o h i@ use
o/192.168.1.105/DATA2_QR01_CD_00_qr01cel03
DATA2_QR01 irs to
s b e
QR01CEL03 NORMAL

i ( ja ens
S
h
oDATA2_QR01 lic
o/192.168.1.105/DATA2_QR01_CD_08_qr01cel03

ir QR01CEL03 NORMAL

Jasb o/192.168.1.105/DATA2_QR01_CD_06_qr01cel03
DATA2_QR01 QR01CEL03 NORMAL

o/192.168.1.105/DATA2_QR01_CD_10_qr01cel03

PATH
------------------------------------------------------------------------------
GNAME FAILGROUP STATE
------------------------------ ------------------------------ --------
DATA2_QR01 QR01CEL03 NORMAL

o/192.168.1.105/DATA2_QR01_CD_09_qr01cel03
DATA2_QR01 QR01CEL03 NORMAL

o/192.168.1.103/DATA2_QR01_CD_03_qr01cel01
DATA2_QR01 QR01CEL01 NORMAL

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 30
o/192.168.1.103/DATA2_QR01_CD_09_qr01cel01
DATA2_QR01 QR01CEL01 NORMAL

o/192.168.1.103/DATA2_QR01_CD_08_qr01cel01
DATA2_QR01 QR01CEL01 NORMAL
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

o/192.168.1.103/DATA2_QR01_CD_05_qr01cel01
DATA2_QR01 QR01CEL01 NORMAL

o/192.168.1.103/DATA2_QR01_CD_06_qr01cel01
DATA2_QR01 QR01CEL01 NORMAL

PATH
------------------------------------------------------------------------------
GNAME FAILGROUP STATE
ble
------------------------------ ------------------------------ --------
fe r a
ans
o/192.168.1.103/DATA2_QR01_CD_11_qr01cel01
n - t r
DATA2_QR01 QR01CEL01
n oNORMAL
a
o/192.168.1.103/DATA2_QR01_CD_04_qr01cel01
) h as ฺ
DATA2_QR01 QR01CEL01
c o m uide NORMAL

e c hฺ ent G
fot Stud
o/192.168.1.103/DATA2_QR01_CD_02_qr01cel01
i n
zy this
DATA2_QR01 QR01CEL01 NORMAL
e
h i@ use
o/192.168.1.103/DATA2_QR01_CD_10_qr01cel01
o
DATA2_QR01
b i rs e to QR01CEL01 NORMAL

( j a s ns
hi lice
o/192.168.1.103/DATA2_QR01_CD_00_qr01cel01
oDATA2_QR01
b i rS QR01CEL01 NORMAL

Ja s o/192.168.1.103/DATA2_QR01_CD_01_qr01cel01
DATA2_QR01 QR01CEL01 NORMAL

PATH
------------------------------------------------------------------------------
GNAME FAILGROUP STATE
------------------------------ ------------------------------ --------
o/192.168.1.103/DATA2_QR01_CD_07_qr01cel01
DATA2_QR01 QR01CEL01 NORMAL

o/192.168.1.105/DATA2_QR01_CD_03_qr01cel03
DATA2_QR01 QR01CEL03 NORMAL

o/192.168.1.105/DATA2_QR01_CD_05_qr01cel03
DATA2_QR01 QR01CEL03 NORMAL

o/192.168.1.105/DATA2_QR01_CD_01_qr01cel03
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 31
DATA2_QR01 QR01CEL03 NORMAL

o/192.168.1.105/DATA2_QR01_CD_04_qr01cel03
DATA2_QR01 QR01CEL03 NORMAL

o/192.168.1.104/DATA2_QR01_CD_09_qr01cel02
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

DATA2_QR01 QR01CEL02 NORMAL

o/192.168.1.104/DATA2_QR01_CD_06_qr01cel02

PATH
------------------------------------------------------------------------------
GNAME FAILGROUP STATE
------------------------------ ------------------------------ --------
DATA2_QR01 QR01CEL02 NORMAL
ble
fe r a
o/192.168.1.104/DATA2_QR01_CD_07_qr01cel02
ans
DATA2_QR01 QR01CEL02
- t
NORMAL
n r
n o
a
has eฺ
o/192.168.1.104/DATA2_QR01_CD_11_qr01cel02
DATA2_QR01 QR01CEL02
) NORMAL

ฺ c om Guid
t e ch dent
o/192.168.1.104/DATA2_QR01_CD_08_qr01cel02

nfo Stu
DATA2_QR01 QR01CEL02 NORMAL

yi
z this
e
o/192.168.1.104/DATA2_QR01_CD_01_qr01cel02
DATA2_QR01
o h i@ use QR01CEL02 NORMAL

s b irs e to
i ( ja ens
o/192.168.1.104/DATA2_QR01_CD_10_qr01cel02

oh
DATA2_QR01
lic QR01CEL02
S o/192.168.1.104/DATA2_QR01_CD_03_qr01cel02
NORMAL

ir
Jasb DATA2_QR01 QR01CEL02 NORMAL

PATH
------------------------------------------------------------------------------
GNAME FAILGROUP STATE
------------------------------ ------------------------------ --------

o/192.168.1.104/DATA2_QR01_CD_00_qr01cel02
DATA2_QR01 QR01CEL02 NORMAL

o/192.168.1.104/DATA2_QR01_CD_04_qr01cel02
DATA2_QR01 QR01CEL02 NORMAL

o/192.168.1.104/DATA2_QR01_CD_02_qr01cel02
DATA2_QR01 QR01CEL02 NORMAL

o/192.168.1.104/DATA2_QR01_CD_05_qr01cel02
DATA2_QR01 QR01CEL02 NORMAL
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 32
36 rows selected.

SQL>
9. Drop the disk group you created in step 7.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

SQL> drop diskgroup data2_qr01;

Diskgroup dropped.

SQL>
10. Exit your SQL*Plus session.

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 33
Practice 6-4: Configuring Exadata Storage Security
Overview
In this practice, you configure Exadata storage security.

Assumptions
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Before beginning this practice you must complete Practice 6-2. Your ability to complete this
practice depends on the existence of the grid disks that are created in practice 6-2.

Tasks
Exadata storage security has two modes; ASM-scoped security and database-scoped security.
ASM-scoped security must be implemented before database-scoped security can be
configured. In the first part of this practice, you will configure ASM-scoped security across your
lab environment.
ble
1. Establish a terminal connection to qr01db01 as the grid user.
fe r a
2. Using SQL*Plus, connect to ASM as an ASM administrator.
ans
[grid@qr01db01 ~]$ sqlplus / as sysasm n - t r
a no
SQL*Plus: Release 11.2.0.3.0 Production...as
m ) h ideฺ
SQL> h ฺ co t Gu
c den
teASM
i n o
3. Note the DB_UNIQUE_NAME setting for fthe
tu
environment and then exit SQL*Plus.
y
z this S
e
SQL> show parameter unique

o h i@ use
NAME
s b irs e to TYPE VALUE
( ja ens
------------------------------------
i
----------- -------------
h li
odb_unique_name c string +ASM
S
a s bir SQL> exit
J Disconnected from Oracle Database 11g Enterprise Edition Release
11.2.0.3.0 - 64bit Production...
[oracle@qr01db01 ~]$
4. Use the su command to assume the privileges of the root user. Enter oracle when
prompted for the password.
[grid@qr01db01 ~]$ su
Password: <oracle>
[root@qr01db01 grid]#
5. Shut down the Oracle Database, ASM, and Grid Infrastructure. Note that on a Database
Machine, you would need to perform this step on every server in the ASM cluster.
[root@qr01db01 grid]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'qr01db01'
CRS-2673: Attempting to stop 'ora.crsd' on 'qr01db01'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on
'qr01db01'
CRS-2673: Attempting to stop 'ora.oc4j' on 'qr01db01'
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 34
CRS-2673: Attempting to stop 'ora.qr01db02.vip' on 'qr01db01'
...
CRS-2673: Attempting to stop 'ora.net1.network' on 'qr01db01'
CRS-2677: Stop of 'ora.net1.network' on 'qr01db01' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'qr01db01'
has completed
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

CRS-2677: Stop of 'ora.crsd' on 'qr01db01' succeeded


CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'qr01db01'
...
CRS-2677: Stop of 'ora.diskmon' on 'qr01db01' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'qr01db01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'qr01db01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@qr01db01 grid]#
ble
6. Exit the root user session.
fe r a
[root@qr01db01 grid]# exit
ans
exit n - t r
[grid@qr01db01 ~]$
a no
) h as terminal
7. Leave the current terminal session active and establish a separate ฺ connection to
e
the qr01cel01 Exadata cell as the celladmin user.m
ฺ c o Guid
8. Launch the Exadata cell command-line interface
e c h ent
(CellCLI).
[celladmin@qr01cel01 ~]$ cellcli t
fo Stud
y i n
CellCLI: Release 11.2.3.2.1
e z th–isProduction...
CellCLI> rso
hi@ use
s b i e to
9. Use the CREATE
( ja KEY e s
ncommand to generate a random hexadecimal key string. Then exit
oh i
CellCLI. li c
ir S
s b CellCLI> create key
Ja aecacf517c96683eb33eaff589a59818
CellCLI> exit
quitting

[celladmin@qr01cel01 ~]$
10. Use the ASSIGN KEY command to assign the security key generated in step 9 to the
Oracle ASM cluster on all the cells that you want the Oracle ASM cluster to access. Use the
DB_UNIQUE_NAME observed earlier. Note that this is case-sensitive.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 "cellcli -e \
> assign key for +ASM=\'aecacf517c96683eb33eaff589a59818\'"
qr01cel01: Key for +ASM successfully created
qr01cel02: Key for +ASM successfully created
qr01cel03: Key for +ASM successfully created
[celladmin@qr01cel01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 35
11. Use the CREATE GRIDDISK or ALTER GRIDDISK command to configure security on the
grid disks you want the Oracle ASM cluster to access. Set the Oracle ASM
DB_UNIQUE_NAME in the availableTo attribute of each grid disk.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 "cellcli -e \
> alter griddisk all availableTo=\'+ASM\'"
qr01cel01: GridDisk DATA2_QR01_CD_00_qr01cel01 successfully altered
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

qr01cel01: GridDisk DATA2_QR01_CD_01_qr01cel01 successfully altered


qr01cel01: GridDisk DATA2_QR01_CD_02_qr01cel01 successfully altered
qr01cel01: GridDisk DATA2_QR01_CD_03_qr01cel01 successfully altered
qr01cel01: GridDisk DATA2_QR01_CD_04_qr01cel01 successfully altered
...
qr01cel03: GridDisk RECO_QR01_CD_07_qr01cel03 successfully altered
qr01cel03: GridDisk RECO_QR01_CD_08_qr01cel03 successfully altered
qr01cel03: GridDisk RECO_QR01_CD_09_qr01cel03 successfully altered
qr01cel03: GridDisk RECO_QR01_CD_10_qr01cel03 successfully altered
ble
qr01cel03: GridDisk RECO_QR01_CD_11_qr01cel03 successfully altered
fe r a
s
ran
[celladmin@qr01cel01 ~]$

12. Return to your grid user session on qr01db01 and change directory to n-t
/etc/oracle/cell/network-config. a no
[grid@qr01db01 ~]$ cd /etc/oracle/cell/network-config
) has eฺ
[grid@qr01db01 network-config]$
ฺ c om Guid
13. Create a cellkey.ora file containing the key
t e e ntstep 9 and the
chvaluedfrom
DB_UNIQUE_NAME for the ASM cluster.fo tu
yi n S
e z thi
[grid@qr01db01 network-config]$ s cat << END > cellkey.ora
h i@ use
> key=aecacf517c96683eb33eaff589a59818
o
> asm=+ASMirs to
s b e
> ENDja
i ( c e ns
h li
o[grid@qr01db01 network-config]$
S
a s birConfirm the contents of the cellkey.ora file.
14.
J [grid@qr01db01 network-config]$ cat cellkey.ora
key=aecacf517c96683eb33eaff589a59818
asm=+ASM
[grid@qr01db01 network-config]$
15. Set the file permissions and verify the settings. Note that on a Database Machine you would
need to configure the cellkey.ora file on every server in the ASM cluster.
[grid@qr01db01 network-config]# chown grid:asmadmin cellkey.ora
[grid@qr01db01 network-config]$ chmod 640 cellkey.ora
[grid@qr01db01 network-config]$ ls -l cellkey.ora
-rw-r----- 1 grid asmadmin 46 Jul 17 20:50 cellkey.ora
[grid@qr01db01 network-config]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 36
16. Use the su command to assume the privileges of the root user. Enter oracle when
prompted for the password.
[grid@qr01db01 network-config]$ su
Password: <oracle>
[root@qr01db01 network-config]#
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

17. Restart the Oracle Database, ASM and Grid Infrastructure. Note that on a Database
Machine you would need to perform this step on every server in the ASM cluster.
[root@qr01db01 network-config]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@qr01db01 network-config]#
18. Verify that all the Oracle cluster resources restart using the following command. Note that
you may receive an error message indicating a failure to communicate with a cluster
service if you execute the command while the cluster is restarting. You can safely ignore
ble
the error message and re-execute the command until all the resources start. Do not
fe r a
proceed to the next step before all the cluster resources restart.
ans
[root@qr01db01 network-config]# crsctl stat res -w "TARGET = ONLINE" -t
n - t r
n
------------------------------------------------------------------------------ o
a
has eฺ
NAME TARGET STATE SERVER STATE_DETAILS

)
------------------------------------------------------------------------------
Local Resources
ฺ c om Guid
t e ch dent
------------------------------------------------------------------------------

nfo Stu
ora.DATA_QR01.dg

yi
ONLINE ONLINE
z this
qr01db01
ora.DBFS_DG.dg e
o h i@ use
ONLINE ONLINE qr01db01

s b irs e to
ora.LISTENER.lsnr

i ( ja ens ONLINE ONLINE qr01db01

S oh lic
ora.RECO_QR01.dg

ir
ONLINE ONLINE qr01db01

asb
ora.asm
J ONLINE ONLINE qr01db01 Started
ora.net1.network
ONLINE ONLINE qr01db01
ora.ons
ONLINE ONLINE qr01db01
------------------------------------------------------------------------------
Cluster Resources
------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE qr01db01
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE qr01db01
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE qr01db01
ora.cvu
1 ONLINE ONLINE qr01db01
ora.dbm.db
1 ONLINE ONLINE qr01db01 Open
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 37
2 ONLINE OFFLINE
ora.oc4j
1 ONLINE ONLINE qr01db01
ora.qr01db01.vip
1 ONLINE ONLINE qr01db01
ora.qr01db02.vip
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

1 ONLINE INTERMEDIATE qr01db01 FAILED OVER


ora.scan1.vip
1 ONLINE ONLINE qr01db01
ora.scan2.vip
1 ONLINE ONLINE qr01db01
ora.scan3.vip
1 ONLINE ONLINE qr01db01

[root@qr01db01 network-config]#
ble
19. Exit the root user session and change back to the grid user home directory.
fe r a
ans
[root@qr01db01 network-config]# exit
n - t r
exit
a no
has eฺ
[grid@qr01db01 network-config]$ cd
[grid@qr01db01 ~]$
)
ASM-scoped security is now configured. The fact that ASM
ฺ c G uidDatabase restarted
omand Oracle
shows that the ASM environment can access the grid
e ch disks e n t
configured on the Exadata storage.
To further prove this is the case, you will nowf o t
create a
t u d
disk group on some of the grid disks.
y i n S
20. Using SQL*Plus, connect to ASMzas an ASM
e t h is administrator.
[grid@qr01db01 ~]$
o h i@sqlplus
u s e / as sysasm
s b irs e to
( ja Release
SQL*Plus:
i c e ns 11.2.0.3.0 Production...
S oh li
ir
asb
SQL>
J 21. Create a new ASM disk group consuming the grid disks created in a previous practice (use
the SQL script /home/oracle/labs/lab06-04-21.sql if you prefer).
SQL> create diskgroup data2_qr01_asm_sec normal redundancy
2 disk 'o/*/DATA2_QR01*'
3 attribute 'compatible.rdbms' = '11.2.0.0.0',
4 'compatible.asm' = '11.2.0.0.0',
5 'cell.smart_scan_capable' = 'TRUE',
6 'au_size' = '4M';

Diskgroup created.

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 38
22. Drop the newly created disk group and exit SQL*Plus.
SQL> drop diskgroup data2_qr01_asm_sec;

Diskgroup dropped.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release
11.2.0.3.0 - 64bit Production...
[grid@qr01db01 ~]$
As expected, the ASM cluster is able to access the Exadata storage using ASM-scoped
security. Now imagine that another ASM cluster is configured and some of the grid disks are
assigned to it. In the next part of this practice you will reconfigure the Exadata storage and see
the effect.
ble
23. Leave the current terminal session active and establish a separate terminal connection to
fe r a
the qr01cel01 Exadata cell as the celladmin user.
ans
24. Launch the Exadata cell command-line interface (CellCLI).
n - t r
[celladmin@qr01cel01 ~]$ cellcli
a no
CellCLI: Release 11.2.3.2.1 – Production...
) has eฺ
ฺ c om Guid
CellCLI>
e h ent
canother
25. Use the CREATE KEY command to generate f o t t u drandom hexadecimal key string. Then
exit CellCLI. yi n S
e z h i s
CellCLI> create ikey
h @ se t
r so to u
4b03b5b2b54c871de54784b8064dabdd
b i
CellCLI>
( j a s exitnse
o hi
quitting
l ice
b i rS
Jas [celladmin@qr01cel01 ~]$
26. Use the ASSIGN KEY command to assign the security key generated in step 25 to another
Oracle ASM cluster called +ASM2.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 "cellcli -e \
> assign key for +ASM2=\'4b03b5b2b54c871de54784b8064dabdd\'"
qr01cel01: Key for +ASM2 successfully created
qr01cel02: Key for +ASM2 successfully created
qr01cel03: Key for +ASM2 successfully created
[celladmin@qr01cel01 ~]$
27. Confirm the key assignment. Note that each cell now has two key assignments for different
ASM clusters.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
> list key
qr01cel01: +ASM aecacf517c96683eb33eaff589a59818
qr01cel01: +ASM2 4b03b5b2b54c871de54784b8064dabdd
qr01cel02: +ASM aecacf517c96683eb33eaff589a59818
qr01cel02: +ASM2 4b03b5b2b54c871de54784b8064dabdd

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 39
qr01cel03: +ASM aecacf517c96683eb33eaff589a59818
qr01cel03: +ASM2 4b03b5b2b54c871de54784b8064dabdd
[celladmin@qr01cel01 ~]$

28. Drop the grid disks having prefix=DATA2_QR01.


[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

> drop griddisk all prefix=DATA2_QR01


qr01cel01: GridDisk DATA2_QR01_CD_00_qr01cel01 successfully dropped
qr01cel01: GridDisk DATA2_QR01_CD_01_qr01cel01 successfully dropped
qr01cel01: GridDisk DATA2_QR01_CD_02_qr01cel01 successfully dropped
qr01cel01: GridDisk DATA2_QR01_CD_03_qr01cel01 successfully dropped
qr01cel01: GridDisk DATA2_QR01_CD_04_qr01cel01 successfully dropped
...
qr01cel03: GridDisk DATA2_QR01_CD_07_qr01cel03 successfully dropped
qr01cel03: GridDisk DATA2_QR01_CD_08_qr01cel03 successfully dropped
ble
qr01cel03: GridDisk DATA2_QR01_CD_09_qr01cel03 successfully dropped
fe r a
qr01cel03: GridDisk DATA2_QR01_CD_10_qr01cel03 successfully dropped
ans
qr01cel03: GridDisk DATA2_QR01_CD_11_qr01cel03 successfully dropped
n - t r
no
[celladmin@qr01cel01 ~]$

a
29. Create a new set of grid disks with the availableTo attribute set to +ASM2.

) has eฺ "cellcli -e \
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03
> create griddisk all harddisk prefix=DATA2_QR01,
ฺ c uid created
omsuccessfully
availableTo=\'+ASM2\'"
G
qr01cel01: GridDisk DATA2_QR01_CD_00_qr01cel01
e t
ch densuccessfully
t
fo Stu successfully created
qr01cel01: GridDisk DATA2_QR01_CD_01_qr01cel01

yi n
qr01cel01: GridDisk DATA2_QR01_CD_02_qr01cel01 created
e z t h i s
qr01cel01: GridDisk DATA2_QR01_CD_03_qr01cel01 successfully created
i @ e
h DATA2_QR01_CD_04_qr01cel01
s
qr01cel01: GridDisk
... i r s o t o u successfully created

a s bGridDisks e
h i (j iGridDisk
qr01cel03:
c en DATA2_QR01_CD_08_qr01cel03 successfully created
DATA2_QR01_CD_07_qr01cel03 successfully created

S
qr01cel03:
l
oqr01cel03: GridDisk DATA2_QR01_CD_09_qr01cel03 successfully created
ir
Jasb qr01cel03: GridDisk DATA2_QR01_CD_10_qr01cel03 successfully created
qr01cel03: GridDisk DATA2_QR01_CD_11_qr01cel03 successfully created
[celladmin@qr01cel01 ~]$

Note that the ALTER GRIDDISK command could have been used instead of dropping and
recreating the grid disks. However, the ALTER GRIDDISK command can only be run
against all the grid disks (regardless of prefix) or it can be used to modify an individual grid
disk. Hence, using the ALTER GRIDDISK command in this case would require 36 separate
commands (or equivalent scripting).
30. Confirm the availableTo attribute setting for all the grid disks.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
> list griddisk attributes name,availableTo
qr01cel01: DATA2_QR01_CD_00_qr01cel01 +ASM2
qr01cel01: DATA2_QR01_CD_01_qr01cel01 +ASM2
qr01cel01: DATA2_QR01_CD_02_qr01cel01 +ASM2
qr01cel01: DATA2_QR01_CD_03_qr01cel01 +ASM2
qr01cel01: DATA2_QR01_CD_04_qr01cel01 +ASM2
...
qr01cel03: RECO_QR01_CD_07_qr01cel03 +ASM
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 40
qr01cel03: RECO_QR01_CD_08_qr01cel03 +ASM
qr01cel03: RECO_QR01_CD_09_qr01cel03 +ASM
qr01cel03: RECO_QR01_CD_10_qr01cel03 +ASM
qr01cel03: RECO_QR01_CD_11_qr01cel03 +ASM
[celladmin@qr01cel01 ~]$
ASM-scoped security is now reconfigured. Now see what happens when +ASM attempts to use
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

the grid disks assigned to +ASM2.


31. Return to your qr01db01 terminal session. Using SQL*Plus, connect to ASM as an ASM
administrator.
[grid@qr01db01 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production...

ble
SQL>
fe r a
32. Create an ASM disk group referencing the grid disks assigned to +ASM2 (use the SQL
ans
script /home/oracle/labs/lab06-04-21.sql if you prefer). Note that the command
n - t r
fails with an error message indicating that the disks do not exist.
a no
SQL> create diskgroup data2_qr01_asm_sec normal
h a s redundancy
2 disk 'o/*/DATA2_QR01*'
m ) i d eฺ
3 attribute 'compatible.rdbms'ฺc G u
=o'11.2.0.0.0',
e h
c den t
4 'compatible.asm' = '11.2.0.0.0',
o t
5 'cell.smart_scan_capable'
y i nf S u
= t'TRUE',
6 'au_size' = '4M'; ez thi s
h i @ s e
create diskgroup
r s o o u
data2_qr01_asm_sec normal redundancy
*
a s bi se t
i
ERROR
h (j at iline
c en 1:
S l
oORA-15018: diskgroup cannot be created
ir
asb
ORA-15031: disk specification 'o/*/DATA2_QR01*' matches no disks
J
SQL>
33. Execute the following query to confirm that the disks are not visible to ASM. Exadata
storage security limits the visibility of grid disks to the environments which are allows to
access them.
SQL> select * from v$asm_disk where path like '%DATA2_QR01%';

no rows selected

SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 41
34. Exit your SQL*Plus session.
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release
11.2.0.3.0 - 64bit Production...
[grid@qr01db01 ~]$
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

So far you have examined ASM-scoped security. Database-scoped security can only be
implemented after ASM-scoped security is already in place. The configuration process for
database-scoped security very similar to the process you have already used for ASM-scoped
security. In the final part of this practice you will configure database-scoped security for one
Oracle Database. In a Database Machine environment with multiple databases you would be
required to repeat the process for each database.
35. Establish a terminal connection to qr01db01 as the oracle user.
36. Using SQL*Plus, connect as a database administrator.
ble
[oracle@qr01db01 ~]$ sqlplus / as sysdba
fe r a
ans
SQL*Plus: Release 11.2.0.3.0 Production...
n - t r
a no
SQL>
) h as ฺ
37. Note the DB_UNIQUE_NAME setting for the database and m then exit e
idSQL*Plus.
c o u
SQL> show parameter unique
e c hฺ ent G
i n f ot tud
NAME z y i s S TYPE VALUE
e
@ se t h
h i
------------------------------------ ----------- -------------
db_unique_name
b i r so to u string dbm
SQL> exit
( j a s nse
h i
Disconnected
l i c e from Oracle Database 11g Enterprise Edition Release
o
b i r S 11.2.0.3.0 - 64bit Production...

Jas
[oracle@qr01db01 ~]$
38. Use the su command to assume the privileges of the root user. Enter oracle when
prompted for the password.
[oracle@qr01db01 ~]$ su
Password: <oracle>
[root@qr01db01 oracle]#
39. Shut down the Oracle Database, ASM and Grid Infrastructure. Note that on a Database
Machine you would need to perform this step on every server in the ASM cluster.
[root@qr01db01 oracle]# /u01/app/11.2.0/grid/bin/crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'qr01db01'
CRS-2673: Attempting to stop 'ora.crsd' on 'qr01db01'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on
'qr01db01'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'qr01db01'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'qr01db01'
...

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 42
CRS-2673: Attempting to stop 'ora.net1.network' on 'qr01db01'
CRS-2677: Stop of 'ora.net1.network' on 'qr01db01' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'qr01db01'
has completed
CRS-2677: Stop of 'ora.crsd' on 'qr01db01' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'qr01db01'
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

...
CRS-2673: Attempting to stop 'ora.gpnpd' on 'qr01db01'
CRS-2677: Stop of 'ora.gpnpd' on 'qr01db01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'qr01db01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@qr01db01 oracle]#

40. Exit the root user session.


[root@qr01db01 oracle]# exit
ble
exit fe r a
[oracle@qr01db01 ~]$
t r a ns
41. Leave the current terminal session active and establish a separate terminal
-
onconnection to
the qr01cel01 Exadata cell as the celladmin user. a n
42. Launch the Exadata cell command-line interface (CellCLI). ha
s
m ) i d eฺ
[celladmin@qr01cel01 ~]$ cellcli co
ฺ G u
e h
c den
CellCLI: Release 11.2.3.2.1 – Production... t
o t
yi nf S tu
CellCLI> ez thi s
h
43. Use the CREATE KEYocommand i @ e
tosgenerate a random hexadecimal key string. Then exit
r s o u
CellCLI.
a s bi se t
i (j icreate
CellCLI>
h c en key
l
SoCellCLI>2877208d48fa273d86ee6492cd6fe331
b i r
Jas
exit
quitting

[celladmin@qr01cel01 ~]$
44. Use the ASSIGN KEY command to assign the security key generated in step 43 to your
Oracle Database on all the cells that you want the database to access. Use the
DB_UNIQUE_NAME observed in step 37. Note that this is case-sensitive.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 "cellcli -e \
> assign key for dbm=\'2877208d48fa273d86ee6492cd6fe331\'"
qr01cel01: Key for dbm successfully created
qr01cel02: Key for dbm successfully created
qr01cel03: Key for dbm successfully created
[celladmin@qr01cel01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 43
45. Use the CREATE GRIDDISK or ALTER GRIDDISK command to configure security on the
grid disks. For database-scoped security you must set the availableTo attribute of each
grid disk to include both the ASM environment and the database which are allowed to
access the grid disk. Use the following command to make all the grid disks accessible to
the dbm database. Note that in other environments you would typically assign groups of grid
disks to different databases.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 "cellcli -e \


> alter griddisk all availableTo=\'+ASM,dbm\'"
qr01cel01: GridDisk DATA2_QR01_CD_00_qr01cel01 successfully altered
qr01cel01: GridDisk DATA2_QR01_CD_01_qr01cel01 successfully altered
qr01cel01: GridDisk DATA2_QR01_CD_02_qr01cel01 successfully altered
qr01cel01: GridDisk DATA2_QR01_CD_03_qr01cel01 successfully altered
qr01cel01: GridDisk DATA2_QR01_CD_04_qr01cel01 successfully altered
...
ble
qr01cel03: GridDisk RECO_QR01_CD_07_qr01cel03 successfully altered
qr01cel03: GridDisk RECO_QR01_CD_08_qr01cel03 successfully altered
fe r a
ans
qr01cel03: GridDisk RECO_QR01_CD_09_qr01cel03 successfully altered
qr01cel03: GridDisk RECO_QR01_CD_10_qr01cel03 successfully altered
n - t r
a
qr01cel03: GridDisk RECO_QR01_CD_11_qr01cel03 successfully altered no
[celladmin@qr01cel01 ~]$
) h as ฺ
46. Return to your oracle user session on qr01db01 and
c o mcreate ide at
a directory
u
$ORACLE_HOME/admin/<DB_UNIQUE_NAME>/pfile
c ฺ nt Gwhere
hdatabase. <DB_UNIQUE_NAME>
represents the DB_UNIQUE_NAME setting for
f o e
t tude Change into the newly created
the
directory.
z yin s S
[oracle@qr01db01 ~]$
i @ e mkdir
e t h-pi $ORACLE_HOME/admin/dbm/pfile
[oracle@qr01db01
s o h ~]$ ucds $ORACLE_HOME/admin/dbm/pfile
b i
[oracle@qr01db01r e to
pfile]$
( s
ja ensfile containing the database key value from step 43 and the
47. Create ai cellkey.ora
S oh
DB_UNIQUE_NAME lic for the ASM cluster associated with the database.
a s bir [oracle@qr01db01 pfile]$ cat << END > cellkey.ora
J > key=2877208d48fa273d86ee6492cd6fe331
> asm=+ASM
> END
[oracle@qr01db01 pfile]$
48. Confirm the contents of the cellkey.ora file.
[oracle@qr01db01 pfile]$ cat cellkey.ora
key=2877208d48fa273d86ee6492cd6fe331
asm=+ASM
[oracle@qr01db01 pfile]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 44
49. Set the file permissions and verify the settings. Note that on a Database Machine you would
need to configure the cellkey.ora file on every database server associated with the
database.
[oracle@qr01db01 pfile]$ chmod 600 cellkey.ora
[oracle@qr01db01 pfile]$ ls -l cellkey.ora
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

-rw------- 1 oracle oinstall 46 Jul 17 21:08 cellkey.ora


[oracle@qr01db01 pfile]$
50. Use the su command to assume the privileges of the root user. Enter oracle when
prompted for the password.
[oracle@qr01db01 pfile]$ su
Password: <oracle>
[root@qr01db01 pfile]#
51. Restart the Oracle Database, ASM and Grid Infrastructure. Note that on a Database ble
Machine you would need to perform this step on every server in the ASM cluster. fe r a
ans
[root@qr01db01 pfile]# /u01/app/11.2.0/grid/bin/crsctl start crs
n - t r
no
CRS-4123: Oracle High Availability Services has been started.
[root@qr01db01 pfile]#
a
52. Verify that all the Oracle cluster resources restart using the h
) as command.
following ฺ Note that
e
you may receive an error message indicating a failure m

service if you execute the command while the cluster c uidYou can safely ignore
to communicate
ois restarting.
G
with a cluster

the error message and re-execute the command


t e chuntildall
e nthet resources start. Do not
fo S
proceed to the next step before thencluster
i tu
restarts.
e y
z thi s
[root@qr01db01 pfile]# /u01/app/11.2.0/grid/bin/crsctl stat res \
> -w "TARGET = ONLINE"
o h i@-t use
s b irsTARGET
e toSTATE
------------------------------------------------------------------------------
NAME
( j a n s SERVER STATE_DETAILS

o hi ice
------------------------------------------------------------------------------
l
r S ------------------------------------------------------------------------------
Local Resources
b i
Jas ora.DATA_QR01.dg
ONLINE ONLINE qr01db01
ora.DBFS_DG.dg
ONLINE ONLINE qr01db01
ora.LISTENER.lsnr
ONLINE ONLINE qr01db01
ora.RECO_QR01.dg
ONLINE ONLINE qr01db01
ora.asm
ONLINE ONLINE qr01db01 Started
ora.net1.network
ONLINE ONLINE qr01db01
ora.ons
ONLINE ONLINE qr01db01
------------------------------------------------------------------------------
Cluster Resources
------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 45
1 ONLINE ONLINE qr01db01
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE qr01db01
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE qr01db01
ora.cvu
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

1 ONLINE ONLINE qr01db01


ora.dbm.db
1 ONLINE ONLINE qr01db01 Open
2 ONLINE OFFLINE
ora.oc4j
1 ONLINE ONLINE qr01db01
ora.qr01db01.vip
1 ONLINE ONLINE qr01db01
ora.qr01db02.vip
ble
1 ONLINE INTERMEDIATE qr01db01 FAILED OVER
fe r a
ora.scan1.vip
ans
1 ONLINE ONLINE qr01db01
n - t r
ora.scan2.vip
a no
as ฺ
1 ONLINE ONLINE qr01db01
ora.scan3.vip
) h
1 ONLINE ONLINE qr01db01 m
c o u ide
e c hฺ ent G
[root@qr01db01 pfile]#
i n f ot tud
e zy tback
53. Exit the root user session and change
h i s toSthe oracle user home directory.
[root@qr01db01 pfile]#
h i @ sexit e
exit o
irs e pfile]$
to u
a s b
[oracle@qr01db01 s cd
( j e n
o hi lic
[oracle@qr01db01 ~]$
S
ir that the bothsecurity
Database-scoped is now configured. The fact that ASM and Oracle Database restarted
s b
Ja shows environments can access the grid disks configured on the Exadata storage.
54. Confirm that the database can access its storage by connecting to the database and
executing a query.
[oracle@qr01db01 ~]$ sqlplus sales/sales

SQL*Plus: Release 11.2.0.3.0 Production...

SQL> select count(*) from customers;

COUNT(*)
----------
1500000

SQL> exit

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 46
Disconnected from Oracle Database 11g Enterprise Edition Release
11.2.0.3.0 - 64bit Production...
[oracle@qr01db01 ~]$
In the final part of the practice you will remove the Exadata storage security that you have
configured during this practice.
55. Use the su command to assume the privileges of the root user.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

[oracle@qr01db01 ~]$ su
Password: <oracle>
[root@qr01db01 oracle]#
56. Shut down the Oracle Database, ASM and Grid Infrastructure.
[root@qr01db01 oracle]# /u01/app/11.2.0/grid/bin/crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'qr01db01'
ble
CRS-2673: Attempting to stop 'ora.crsd' on 'qr01db01'
fe r a
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on
ans
'qr01db01'
n - t r
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'qr01db01'
n o
a
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'qr01db01'
...
) has eฺ
om Guid
CRS-2673: Attempting to stop 'ora.net1.network' on 'qr01db01'

ฺ c
CRS-2677: Stop of 'ora.net1.network' on 'qr01db01' succeeded

e ch dent
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'qr01db01'
t
nfo Stu
has completed
yi
z this
CRS-2677: Stop of 'ora.crsd' on 'qr01db01' succeeded
e
...
o h i@ use
CRS-2673: Attempting to stop 'ora.ctssd' on 'qr01db01'

b irs e to
CRS-2673: Attempting to stop 'ora.gpnpd' on 'qr01db01'
s
( ja ens
CRS-2677: Stop of 'ora.gpnpd' on 'qr01db01' succeeded
i
S oh lic
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on

ir 'qr01db01' has completed

asb
CRS-4133: Oracle High Availability Services has been stopped.
J [root@qr01db01 oracle]#

57. Back in your celladmin session on qr01cel01, use the following command to clear the
availableTo grid disk attribute.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 "cellcli -e \
> alter griddisk all availableTo=\'\'"
qr01cel01: GridDisk DATA2_QR01_CD_00_qr01cel01 successfully altered
qr01cel01: GridDisk DATA2_QR01_CD_01_qr01cel01 successfully altered
qr01cel01: GridDisk DATA2_QR01_CD_02_qr01cel01 successfully altered
qr01cel01: GridDisk DATA2_QR01_CD_03_qr01cel01 successfully altered
qr01cel01: GridDisk DATA2_QR01_CD_04_qr01cel01 successfully altered
...
qr01cel03: GridDisk RECO_QR01_CD_07_qr01cel03 successfully altered
qr01cel03: GridDisk RECO_QR01_CD_08_qr01cel03 successfully altered
qr01cel03: GridDisk RECO_QR01_CD_09_qr01cel03 successfully altered
qr01cel03: GridDisk RECO_QR01_CD_10_qr01cel03 successfully altered
qr01cel03: GridDisk RECO_QR01_CD_11_qr01cel03 successfully altered
[celladmin@qr01cel01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 47
58. Clear the key assignment for the dbm database.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 "cellcli -e \
> assign key for dbm=\'\'"
qr01cel01: Key for dbm successfully dropped
qr01cel02: Key for dbm successfully dropped
qr01cel03: Key for dbm successfully dropped
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

[celladmin@qr01cel01 ~]$

59. Clear the key assignments for +ASM and +ASM2.


[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 "cellcli -e \
> assign key for +ASM=\'\'"
qr01cel01: Key for +ASM successfully dropped
qr01cel02: Key for +ASM successfully dropped
qr01cel03: Key for +ASM successfully dropped
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 "cellcli -e \
ble
> assign key for +ASM2=\'\'"
fe r a
qr01cel01: Key for +ASM2 successfully dropped
ans
qr01cel02: Key for +ASM2 successfully dropped
n - t r
qr01cel03: Key for +ASM2 successfully dropped
[celladmin@qr01cel01 ~]$ a no
60. Return to your root session and remove the cellkey.ora
) as that ฺyou created earlier in
hfiles
the practice.
c o m uide
c hฺ ent G
[root@qr01db01 oracle]# rm $ORACLE_HOME/admin/dbm/pfile/cellkey.ora
e
rm: remove regular file
i n fot Stud
zy/etc/oracle/cell/network-config/cellkey.ora
`/u01/app/oracle/product/11.2.0/dbhome_1/admin/dbm/pfile/cellkey.ora'? y
[root@qr01db01 oracle]# e rm
th i s
i
rm: remove regularhfile @ `/etc/oracle/cell/network-config/cellkey.ora'?
s e
[root@qr01db01
o
irsoracle]#
to u y

s b e
61. Restart the
i ( c e ns ASM and Grid Infrastructure.
jaOracle Database,
h li pfile]# /u01/app/11.2.0/grid/bin/crsctl start crs
o[root@qr01db01
ir S
s b CRS-4123: Oracle High Availability Services has been started.

Ja [root@qr01db01 pfile]#
62. Exit all of your terminal sessions.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 48
Practice 6-5: Cell User Accounts
Overview
In this practice, you exercise the privileges available to the celladmin and cellmonitor
Exadata administration accounts.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Tasks
1. Establish a terminal connection to the qr01cel01 Exadata cell as the cellmonitor user.
Enter welcome as the password when prompted.
2. Launch the Exadata cell command-line interface (CellCLI).
[cellmonitor@qr01cel01 ~]$ cellcli
CellCLI: Release 11.2.3.2.1 – Production...

ble
CellCLI>
fe r a
The cellmonitor user can only view Exadata cell objects using the CellCLI LIST command.
ans
3. Confirm that cellmonitor can view the Exadata cell attributes. n - t r
CellCLI> list cell detail
a no
name: qr01cel01
) h as ฺ
bbuTempThreshold: 60
c o m uide
bbuChargeThreshold: 800

e c hฺ ent G
bmcType:
t tud
absent
foOSS_11.2.3.2.1_LINUX.X64_130109
cellVersion:
i n
cpuCount:
e zy th1 is S
diagHistoryDays:
h i @ se 7
fanCount:
i r s o to u 1/1

( j a sb nse
fanStatus: normal

h i l i c e
flashCacheMode: WriteThrough

S o id: 8ab50138-a667-4793-a976-c540dc1930c5
ir
asb
interconnectCount: 3

J interconnect1:
iormBoost:
eth1
0.0
ipaddress1: 192.168.1.103/24
kernelVersion: 2.6.32-400.11.1.el5uek
makeModel: Fake hardware
metricHistoryDays: 7
notificationMethod: mail
notificationPolicy: critical,warning,clear
offloadEfficiency: 596.0
powerCount: 1/1
powerStatus: normal
releaseVersion: 11.2.3.2.1
releaseTrackingBug: 14522699
smtpFrom: "John Doe"
smtpFromAddr: john.doe@example.com
smtpServer: my_mail.example.com
smtpToAddr: jane.smith@example.com
status: online

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 49
temperatureReading: 0.0
temperatureStatus: normal
upTime: 0 days, 2:45
cellsrvStatus: running
msStatus: running
rsStatus: running
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

CellCLI>

4. Confirm that cellmonitor cannot modify the Exadata cell attributes.


CellCLI> alter cell smtpToAddr='admin@example.com'
CELL-01520: This command is not permitted in monitor mode.

CellCLI>
ble
5. Confirm that cellmonitor cannot create or modify the Exadata cell objects. fe r a
ans
CellCLI> create celldisk all harddisk
n - t r
CELL-01520: This command is not permitted in monitor mode.
a no
asa comment"
his
CellCLI> alter griddisk all comment="Here
m ) i d eฺ
CELL-01520: This command is not permitted
h ฺ co t Ginu monitor mode.
t e c den
i nf o tu
CellCLI>
y
z thi s S
6. Establish a terminal connectioneto the qr01cel01 Exadata cell as the celladmin user.
h i @ e
7. Launch the Exadata cell
i r s t o us interface (CellCLI).
o command-line
( j a sb nse ~]$ cellcli
[celladmin@qr01cel01

o hi
CellCLI:
l ce
iRelease 11.2.3.2.1 – Production...

b i r S CellCLI>
Jas In previous practices you have already seen how the celladmin user can create, modify and
drop Exadata cell objects. In fact, the celladmin user can execute any CellCLI command
except for the CALIBRATE command. The CALIBRATE command can only be executed by the
root user.
8. Confirm that celladmin cannot run the CALIBRATE command.
CellCLI> calibrate
CELL-01522: CALIBRATE must be run as the root user id.

CellCLI>
9. Exit your CellCLI sessions.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 50
Practice 6-6: Using the Distributed Command-Line Utility (dcli)

Overview
The distributed command-line utility (dcli) is a utility program that is provided with Database
Machine. Its purpose is to provide a means to simultaneously execute monitoring and
administration commands across multiple servers.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

In earlier practices you used dcli to execute CellCLI commands across multiple Exadata cells.
In this practice you will extend your use of dcli by performing the initial configuration required
to enable dcli to issue commands to all of your cells from your database server (qr01db01).
You will also exercise some additional dcli functions.

Tasks
1. Establish a terminal connection to qr01db01 as the oracle user.
2. Create a file named mycells that contains the names of your Exadata cells on separate ble
lines. fe r a
ans
[oracle@qr01db01 ~]$ cat << END > mycells
n - t r
> qr01cel01
a no
has eฺ
> qr01cel02
> qr01cel03
)
> END
ฺ c om Guid
[oracle@qr01db01 ~]$
t e ch dent
3. Generate a private/public key pair forin fowith S
use tuusing the ssh-keygen command as
SSH
y
ez taccept s
shown below. For the sake of simplicity,
when you are promptedh i
for@ e hi the default key file and just press Enter
a passphrase.
i r s t o us -t rsa
o~]$ ssh-keygen
sbpublic/private
[oracle@qr01db01

( j a
Generating
e n se rsa key pair.
hi fileliinc which to save the key (/home/oracle/.ssh/id_rsa):
oEnter
S
bir Enter same passphrase again:
Enter passphrase (empty for no passphrase):

J a s Your identification has been saved in /home/oracle/.ssh/id_rsa.


Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
26:ee:bd:76:6c:33:20:3f:8b:9f:23:49:77:6a:15:97 oracle@qr01db01
[oracle@qr01db01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 51
4. Execute the following command to configure SSH user-equivalence between your database
server OS account (oracle) and the celladmin user on the cells specified in the
mycells file.
Answer yes if you are prompted to acknowledge server authenticity.
[oracle@qr01db01 ~]$ dcli -g mycells -k
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

The authenticity of host 'qr01cel01 (192.0.2.103)' can't be established.


RSA key fingerprint is fb:f9:ec:56:6b:8c:5b:a0:90:82:20:36:51:b8:59:af.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'qr01cel01,192.0.2.103' (RSA) to the list of known
hosts.
celladmin@qr01cel01's password: <welcome>
The authenticity of host 'qr01cel03 (192.0.2.105)' can't be established.
RSA key fingerprint is fb:f9:ec:56:6b:8c:5b:a0:90:82:20:36:51:b8:59:af.
Are you sure you want to continue connecting (yes/no)? yes
ble
Warning: Permanently added 'qr01cel03,192.0.2.105' (RSA) to the list of known
hosts.
fe r a
celladmin@qr01cel03's password: <welcome>
ans
The authenticity of host 'qr01cel02 (192.0.2.104)' can't be established.
n - t r
no
RSA key fingerprint is fb:f9:ec:56:6b:8c:5b:a0:90:82:20:36:51:b8:59:af.
a
has eฺ
Are you sure you want to continue connecting (yes/no)? yes

)
Warning: Permanently added 'qr01cel02,192.0.2.104' (RSA) to the list of known
hosts.
ฺ c om Guid
t e ch dent
celladmin@qr01cel02's password: <welcome>

nfo Stu
qr01cel01: ssh key added
qr01cel02: ssh key added
yi
z this
qr01cel03: ssh key added e
h
[oracle@qr01db01 ~]$
o i@ use
You have now completed
s b irsthe eone-time
to configuration required to enable dcli between the
oracle user on
i ( jaqr01db01
c e nsand the celladmin user on each Exadata cell. After dcli is
executed h with the
oonce li –k option, subsequent dcli commands between the same servers and
ir
user S
accounts do not require the –k option and do not require a password.
s b
Ja 5. The primary use for dcli is to simultaneously execute CellCLI commands across multiple
cells. Use the following command to check on the status of your Exadata cells.
[oracle@qr01db01 ~]$ dcli -g mycells cellcli -e list cell
qr01cel01: qr01cel01 online
qr01cel02: qr01cel02 online
qr01cel03: qr01cel03 online
[oracle@qr01db01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 52
6. dcli can also be used to execute any non-interactive operating system commands on
multiple cells and/or database servers. You can use quotes to surround compound
commands and commands which contain pipes. Execute the following example command
or construct an alternative command of your own.
[oracle@qr01db01 ~]$ dcli -g mycells "cat /proc/meminfo | grep Mem"
qr01cel01: MemTotal: 1993500 kB
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

qr01cel01: MemFree: 62564 kB


qr01cel02: MemTotal: 1993500 kB
qr01cel02: MemFree: 71484 kB
qr01cel03: MemTotal: 1993500 kB
qr01cel03: MemFree: 73124 kB
[oracle@qr01db01 ~]$

7. dcli is not just limited to monitoring. It is often used to ensure that consistent settings are
applied across multiple systems. Use the following commands to view and adjust IORM
settings on your Exadata cells. ble
fe r a
[oracle@qr01db01 ~]$ dcli -g mycells cellcli -e \
ans
> list iormplan attributes objective
n - t r
qr01cel01: basic
qr01cel02: basic a no
qr01cel03: basic
) h as ฺ
c o m
[oracle@qr01db01 ~]$ dcli -g mycells cellcli
ide objective=auto
-e alter iormplan
u
e c hฺ ent G
qr01cel01: IORMPLAN successfully altered

fot Stud
qr01cel02: IORMPLAN successfully altered

i n
qr01cel03: IORMPLAN successfully altered

e zy this
[oracle@qr01db01 ~]$ dcli -g mycells cellcli
-e \

h i@ use
> list iormplan attributes objective
o
s b irs e to
qr01cel01: auto

ja ens
qr01cel02: auto
i (
oh lic
qr01cel03: auto

ir S [oracle@qr01db01 ~]$ dcli -g mycells cellcli


-e alter iormplan objective=basic

asb
qr01cel01: IORMPLAN successfully altered
J qr01cel02: IORMPLAN successfully altered
qr01cel03: IORMPLAN successfully altered
[oracle@qr01db01 ~]$ dcli -g mycells cellcli -e \
> list iormplan attributes objective
qr01cel01: basic
qr01cel02: basic
qr01cel03: basic
[oracle@qr01db01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 53
8. Sometimes dcli commands return a lot of output. The following command uses wildcards
and a WHERE condition to return the current metric observations for small write I/O requests
for every disk-based cell disk across all the cells. Execute the command and examine the
output for your cells.
[oracle@qr01db01 ~]$ dcli -g mycells "cellcli -e list metriccurrent \
> where name like \'CD_IO_RQ_W_S.?\' and metricobjectname like \'CD.*\'"
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

qr01cel01: CD_IO_RQ_W_SM CD_00_qr01cel01 13,855 IO requests


qr01cel01: CD_IO_RQ_W_SM CD_01_qr01cel01 1,393 IO requests
qr01cel01: CD_IO_RQ_W_SM CD_02_qr01cel01 1,424 IO requests
qr01cel01: CD_IO_RQ_W_SM CD_03_qr01cel01 2,965 IO requests
qr01cel01: CD_IO_RQ_W_SM CD_04_qr01cel01 4,364 IO requests
qr01cel01: CD_IO_RQ_W_SM CD_05_qr01cel01 1,624 IO requests
qr01cel01: CD_IO_RQ_W_SM CD_06_qr01cel01 1,049 IO requests
qr01cel01: CD_IO_RQ_W_SM CD_07_qr01cel01 3,648 IO requests
ble
qr01cel01: CD_IO_RQ_W_SM
qr01cel01: CD_IO_RQ_W_SM
CD_08_qr01cel01
CD_09_qr01cel01
1,567 IO requests
2,352 IO requests
fe r a
ans
qr01cel01: CD_IO_RQ_W_SM
qr01cel01: CD_IO_RQ_W_SM
CD_10_qr01cel01
CD_11_qr01cel01
1,323 IO requests
1,591 IO requests
n - t r
n o
qr01cel02: CD_IO_RQ_W_SM
a
CD_00_qr01cel02 14,336 IO requests

has eฺ
qr01cel02: CD_IO_RQ_W_SM CD_01_qr01cel02 4,056 IO requests
)
om Guid
qr01cel02: CD_IO_RQ_W_SM CD_02_qr01cel02 846 IO requests
qr01cel02: CD_IO_RQ_W_SM
ฺ c
CD_03_qr01cel02 1,732 IO requests
qr01cel02: CD_IO_RQ_W_SM
t e ch dent
CD_04_qr01cel02 3,031 IO requests
qr01cel02: CD_IO_RQ_W_SM
yi nfo StuCD_05_qr01cel02 1,150 IO requests

e z this
qr01cel02: CD_IO_RQ_W_SM CD_06_qr01cel02 1,862 IO requests

o h i@ use
qr01cel02: CD_IO_RQ_W_SM CD_07_qr01cel02 1,506 IO requests

irs e to
qr01cel02: CD_IO_RQ_W_SM CD_08_qr01cel02 3,420 IO requests

s b
qr01cel02: CD_IO_RQ_W_SM CD_09_qr01cel02 1,283 IO requests

i ( ja ens
qr01cel02: CD_IO_RQ_W_SM CD_10_qr01cel02 3,313 IO requests

S oh lic
qr01cel02: CD_IO_RQ_W_SM CD_11_qr01cel02 2,835 IO requests

ir qr01cel03: CD_IO_RQ_W_SM CD_00_qr01cel03 13,564 IO requests

Jasb qr01cel03: CD_IO_RQ_W_SM


qr01cel03: CD_IO_RQ_W_SM
CD_01_qr01cel03
CD_02_qr01cel03
590 IO requests
4,480 IO requests
qr01cel03: CD_IO_RQ_W_SM CD_03_qr01cel03 4,783 IO requests
qr01cel03: CD_IO_RQ_W_SM CD_04_qr01cel03 3,063 IO requests
qr01cel03: CD_IO_RQ_W_SM CD_05_qr01cel03 1,074 IO requests
qr01cel03: CD_IO_RQ_W_SM CD_06_qr01cel03 695 IO requests
qr01cel03: CD_IO_RQ_W_SM CD_07_qr01cel03 5,879 IO requests
qr01cel03: CD_IO_RQ_W_SM CD_08_qr01cel03 562 IO requests
qr01cel03: CD_IO_RQ_W_SM CD_09_qr01cel03 1,449 IO requests
qr01cel03: CD_IO_RQ_W_SM CD_10_qr01cel03 1,269 IO requests
qr01cel03: CD_IO_RQ_W_SM CD_11_qr01cel03 547 IO requests
[oracle@qr01db01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 54
9. This command is essentially the same as the command you just executed in step 8. It uses
the –r option along with a regular expression string to specify which output dcli should
delete from the output. The result is that only the output that does not match the –r regular
expression is returned to the user. Execute the command and examine the output for your
cells.
[oracle@qr01db01 ~]$ dcli -g mycells -r '.*CD_0.*' "cellcli -e list \
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

> metriccurrent where name like \'CD_IO_RQ_W_S.?\' \


> and metricobjectname like \'CD.*\'"
.*CD_0.*: ['qr01cel01', 'qr01cel02', 'qr01cel03']
qr01cel01: CD_IO_RQ_W_SM CD_10_qr01cel01 1,323 IO requests
qr01cel01: CD_IO_RQ_W_SM CD_11_qr01cel01 1,591 IO requests
qr01cel02: CD_IO_RQ_W_SM CD_10_qr01cel02 3,313 IO requests
qr01cel02: CD_IO_RQ_W_SM CD_11_qr01cel02 2,835 IO requests
qr01cel03: CD_IO_RQ_W_SM CD_10_qr01cel03 1,269 IO requests
ble
qr01cel03: CD_IO_RQ_W_SM
[oracle@qr01db01 ~]$
CD_11_qr01cel03 547 IO requests

fe r a
ans
10. While the –r option specifies what to restrict from the dcli output, there is no dcli option
to explicitly define the output that should be returned from a long list. To achieve this aim n - t r
a no
use the grep command in conjunction with dcli. Execute the following command and
examine the output for your cells.
) h as ฺ
[oracle@qr01db01 ~]$ dcli -g mycells "cellclim
o u i de
-e list metriccurrent \
> where name like \'CD_IO_RQ_W_S.?\' \ ฺ c
h ent G
> and metricobjectname like \'CD.*\' e c
t tud| grep CD_00"
qr01cel01: CD_IO_RQ_W_SM
i n foCD_00_qr01cel01
S 14,133 IO requests
qr01cel02: CD_IO_RQ_W_SM zy i s
e t h
CD_00_qr01cel02 14,571 IO requests

h
qr01cel03: CD_IO_RQ_W_SM
o i@ use CD_00_qr01cel03 13,663 IO requests
[oracle@qr01db01
s b irs e to
~]$

dcli can also be s files to numerous remote systems. You will exercise this
(ja usedcetoncopy
in ithe nextliseries
capability h of steps.
S o
11.irCreate a small text file that contains a short message identifying you. Name the file
s b
Ja according to your assigned student account. Confirm the existence and contents of the file.
[oracle@qr01db01 ~]$ cat << END > message.txt
> Hello World!
> END
[oracle@qr01db01 ~]$ cat message.txt
Hello World!
[oracle@qr01db01 ~]$
12. Use dcli with the –f option to copy your file to the default home directory of the
celladmin user on your Exadata cells.
[oracle@qr01db01 ~]$ dcli -g mycells -f message.txt
[oracle@qr01db01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 55
13. Use the following command to confirm the success of the operation in step 12.
[oracle@qr01db01 ~]$ dcli -g mycells cat message.txt
qr01cel01: Hello World!
qr01cel02: Hello World!
qr01cel03: Hello World!
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

[oracle@qr01db01 ~]$
In addition to copying a file to multiple remote locations, dcli can copy a file and execute it
simultaneously on the specified remote systems. You will exercise this capability in the next
series of steps.
14. Create a simple shell script such as the one shown below. Name the file according to your
assigned student account.
[oracle@qr01db01 ~]$ cat << END > script.sh
> HST=\`hostname -s\`
ble
> DTE=\`date\`
fe r a
ans
> echo -n \`cat message.txt\`
n - t r
no
> echo " on \${HST} at \${DTE}."
> END a
[oracle@qr01db01 ~]$
) has eฺ
15. Use the chmod command to make your newly created c om id
script fileuexecutable.
[oracle@qr01db01 ~]$ chmod +xescript.sh

ch dent G
t
[oracle@qr01db01 ~]$
y i nfo Stu
z thrun
16. Use dcli with the –x option toecopy and is the script you just created.
[oracle@qr01db01o~]$h i@dcli u s emycells -x script.sh
irs World!
-g
qr01cel01:bHello
s e toon qr01cel01 at Wed Jul 17 21:27:28 EDT 2013.
i ( ja Hello
qr01cel02:
c e nsWorld! on qr01cel02 at Wed Jul 17 21:27:29 EDT 2013.
h
oqr01cel03: liHello World! on qr01cel03 at Wed Jul 17 21:27:29 EDT 2013.
ir S
s b [oracle@qr01db01 ~]$

Ja Note that script files with the .scl extension are run by the CellCLI utility on the remote
server.
17. Use dcli in conjunction with rm to delete the files you copied to your cells during this
practice. Please be careful not to mistakenly delete any other files.
[oracle@qr01db01 ~]$ dcli -g mycells rm message.txt script.sh
[oracle@qr01db01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Exadata Storage Server Configuration


Chapter 6 - Page 56
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
s
haLesson
Practices )for
m i d eฺ 7: I/O
Resource
ฺ co tManagement
G u
e h
c 7 den
o t
Chapter
y i nf S tu
ez thi s
h i @ s e
o
irs e to u
s b
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 7: I/O Resource Management


Chapter 7 - Page 1
Practices for Lesson 7
Practices Overview
There is no practice for Lesson 3.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 7: I/O Resource Management


Chapter 7 - Page 2
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
s
haLesson
Practices )for
m i d eฺ 8:
Recommendations
ฺ co t Gu for
h
c den Database
Optimizing
e
o t
y i nf Performance
S tu
ez thi s
h i @ s e Chapter 8
o
irs e to u
s b
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 1
Practices for Lesson 8
Practices Overview
In these practices, you will explore the following performance optimization techniques and
technologies:
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

• Configuring write back flash cache


• Using Exadata Hybrid Columnar Compression
• Testing index elimination

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 2
Practice 8-1: Configuring Write Back Flash Cache
Overview
In this practice you will reconfigure Exadata Smart Flash Cache so that write operations can be
serviced by flash only, instead of using disks. This mode of operation is known as write back
flash cache.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Tasks
Reconfiguring Exadata Storage Servers to enable write back flash cache can be achieved
in a rolling manner (one cell at a time) or all-at-once. In this practice you will reconfigure all
the cells at once, which requires that Oracle Database and Grid Infrastructure are shut
down on all database servers. To configure write back flash cache in a rolling manner some
additional steps and checks are recommended. Refer to My Oracle Support bulletin
1500257.1 for details.
1. Establish a terminal connection to qr01db01 as the grid user. ble
fe r a
2. Use the su command to assume the privileges of the root user. Enter oracle when
ans
prompted for the password.
n - t r
[grid@qr01db01 ~]$ su
a no
has eฺ
Password: <oracle>
)
uid that on a Database
[root@qr01db01 grid]#
ฺ c om GNote
3. Shut down the Oracle Database, ASM and Grid Infrastructure.
Machine you would need to perform this stepe conheverye n t in the ASM cluster. Do not
server
t tud
y i nfocompletes.
proceed to the next step until this step
S
e
[root@qr01db01 grid]# crsctl is
z stopthcrs
h i@ usofe Oracle High Availability Services-managed
CRS-2791: Starting shutdown
o
CRS-2673: b
s irs e totostop 'ora.crsd' on 'qr01db01'
resources on 'qr01db01'

ns shutdown of Cluster Ready Services-managed resources on


Attempting

i ( ja Starting
e
CRS-2790:
h
o'qr01db01' li c
S
ir CRS-2673: Attempting to
s b CRS-2673: Attempting stop 'ora.oc4j' on 'qr01db01'

Ja ...
to stop 'ora.qr01db02.vip' on 'qr01db01'

CRS-2673: Attempting to stop 'ora.net1.network' on 'qr01db01'


CRS-2677: Stop of 'ora.net1.network' on 'qr01db01' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'qr01db01'
has completed
CRS-2677: Stop of 'ora.crsd' on 'qr01db01' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'qr01db01'
...
CRS-2677: Stop of 'ora.diskmon' on 'qr01db01' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'qr01db01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'qr01db01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@qr01db01 grid]#

4. Leave the root terminal session active and establish a separate terminal connection to the
qr01cel01 Exadata cell as the celladmin user.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 3
5. Examine the cell flashCacheMode attribute setting on all of the cells. By default, each cell
is configured in WriteThrough mode, which means that write operations must be
persisted to disk regardless of whether or not the data resides inside Exadata Smart Flash
Cache.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
> list cell attributes flashCacheMode
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

qr01cel01: WriteThrough
qr01cel02: WriteThrough
qr01cel03: WriteThrough
[celladmin@qr01cel01 ~]$
6. Drop the existing Exadata Smart Flash Cache.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
> drop flashcache
qr01cel01: Flash cache qr01cel01_FLASHCACHE successfully dropped
ble
qr01cel02: Flash cache qr01cel02_FLASHCACHE successfully dropped
fe r a
qr01cel03: Flash cache qr01cel03_FLASHCACHE successfully dropped
ans
[celladmin@qr01cel01 ~]$
n - t r
7. Stop cellsrv on all of the cells.
n o
s a
) h a
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03

cellcli -e \

ide
> alter cell shutdown services cellsrv
qr01cel01:
c o m u
qr01cel01: Stopping CELLSRV services...hฺ
c n t G
qr01cel01: The SHUTDOWN of CELLSRV e
fot services
u e successful.
dwas
i n S t
qr01cel02:
e zyservices...
h i s
t services was successful.
h @ of CELLSRV
qr01cel02: Stopping CELLSRV
i s e
o to u
qr01cel02: The SHUTDOWN
qr01cel03: irs
a b seCELLSRV services...
s Stopping
( j
qr01cel03:
e n
hi
oqr01cel03: c SHUTDOWN of CELLSRV services was successful.
liThe
b i rS [celladmin@qr01cel01 ~]$

Jas8. Commencing with Exadata storage server release 11.2.3.2.0, the flashCacheMode
attribute can be set to WriteBack, which means that write operations can be serviced by
flash instead of using disks. Enable write back flash cache on all of the cells.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
> alter cell flashCacheMode = WriteBack
qr01cel01: Cell qr01cel01 successfully altered
qr01cel02: Cell qr01cel02 successfully altered
qr01cel03: Cell qr01cel03 successfully altered
[celladmin@qr01cel01 ~]$

9. Restart cellsrv on all of the cells.


[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
> alter cell startup services cellsrv
qr01cel01:
qr01cel01: Starting CELLSRV services...
qr01cel01: The STARTUP of CELLSRV services was successful.
qr01cel02:
qr01cel02: Starting CELLSRV services...

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 4
qr01cel02: The STARTUP of CELLSRV services was successful.
qr01cel03:
qr01cel03: Starting CELLSRV services...
qr01cel03: The STARTUP of CELLSRV services was successful.
[celladmin@qr01cel01 ~]$
10. Re-create Exadata Smart Flash Cache on all the cells.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \


> create flashcache all
qr01cel01: Flash cache qr01cel01_FLASHCACHE successfully created
qr01cel02: Flash cache qr01cel02_FLASHCACHE successfully created
qr01cel03: Flash cache qr01cel03_FLASHCACHE successfully created
[celladmin@qr01cel01 ~]$

11. Verify the cell flashCacheMode attribute setting on all of the cells.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
ble
> list cell attributes flashCacheMode
fe r a
qr01cel01: WriteBack
ans
qr01cel02: WriteBack
n - t r
qr01cel03: WriteBack
a no
as updated
[celladmin@qr01cel01 ~]$
Now that write back flash cache is enabled, each cache may ) hcontain e ฺ data that is
different from the data copy held on disk. This is called o m u
dirty data, i d
and you can monitor the
amount of dirty data inside Exadata Smart Flash c h ฺc nusing
Cache t Gcell metrics.
12. Use the following command to view thefamount
e
ot oftudirty dedata currently inside Exadata Smart
i n Shave only just been created and no
Flash Cache on each cell. Because
databases are currently using e
zythe caches
h
the cells, ttheisamount should be zero.
h i @ e
[celladmin@qr01cel01
i r s o ~]$t o us -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
dcli

( j a sbFC_BY_DIRTY
> list metriccurrent
n se FLASHCACHE
FC_BY_DIRTY

h i
qr01cel01:
l i c e 0.000 MB

S oqr01cel03: FC_BY_DIRTY FLASHCACHE


qr01cel02: FC_BY_DIRTY FLASHCACHE 0.000 MB

s b ir [celladmin@qr01cel01 ~]$ 0.000 MB

Ja 13. Return to you root terminal session and restart Grid Infrastructure, ASM and Oracle
Database.
[root@qr01db01 grid]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@qr01db01 grid]#

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 5
14. Wait for a few minutes until your database restarts. Execute the following command to
monitor the status of the database. Note that you may see an error similar to that shown
below if you execute this command before clusterware is restarted. Proceed to the next
stop only after your database is started on qr01db01.
[root@qr01db01 grid]# srvctl status database -d dbm
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

PRCD-1027 : Failed to retrieve database dbm


PRCR-1115 : Failed to find entities of type resource that match
filters ((NAME == ora.dbm.db) && (TYPE == ora.database.type))
and contain attributes VERSION,ORACLE_HOME,DATABASE_TYPE
Cannot communicate with crsd
[root@qr01db01 grid]# srvctl status database -d dbm
Instance dbm1 is not running on node qr01db01
Instance dbm2 is not running on node qr01db02
ble
[root@qr01db01 grid]# srvctl status database -d dbm
fe r a
Instance dbm1 is running on node qr01db01
ans
Instance dbm2 is not running on node qr01db02
n - t r
[root@qr01db01 grid]#
a no
15. Return to your celladmin terminal session and view the amount
cache. ) has ofedirty

data in each cell

ฺ c om Guid
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
c h n t
> list metriccurrent FC_BY_DIRTY
f o te tu0.961 de MB
qr01cel01: FC_BY_DIRTY
i n
FLASHCACHE
y S 0.961 MB
qr01cel02: FC_BY_DIRTY
e z
FLASHCACHE
t h i s
qr01cel03: FC_BY_DIRTY
h i @ e
FLASHCACHE
s
0.922 MB

b i r
[celladmin@qr01cel01
so to u~]$
At this point, you
( j a e sewould see
s havenconfigured write back flash cache and you have confirmed that it is

o hiyour databases.
being used.
against
Over
l time
ic you more dirty data reported in the cache as you transact

b i rInSthe final part of this practice, you will reconfigure the cells once more and revert back to
Jas write through flash cache.
16. Return to you root terminal session and shut down the Oracle Database, ASM, and Grid
Infrastructure. Do not proceed to the next step until this step completes.
[root@qr01db01 grid]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'qr01db01'
CRS-2673: Attempting to stop 'ora.crsd' on 'qr01db01'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on
'qr01db01'
CRS-2673: Attempting to stop 'ora.oc4j' on 'qr01db01'
CRS-2673: Attempting to stop 'ora.qr01db02.vip' on 'qr01db01'
...
CRS-2673: Attempting to stop 'ora.net1.network' on 'qr01db01'
CRS-2677: Stop of 'ora.net1.network' on 'qr01db01' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'qr01db01'
has completed
CRS-2677: Stop of 'ora.crsd' on 'qr01db01' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'qr01db01'

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 6
...
CRS-2677: Stop of 'ora.diskmon' on 'qr01db01' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'qr01db01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'qr01db01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

[root@qr01db01 grid]#
Switching back to write through flash cache involves essentially the same steps as
switching to write back flash cache. However, because write back flash cache is currently
enabled note that you must first flush the flash cache before dropping it. Flushing Exadata
Smart Flash Cache ensures that all modified data is written to disk.
17. Flush the write back flash cache on all of the cells. Note that the flush operation may take a
few minutes to complete. If you receive a message indicating that the flush operation timed
out for some of the flash disks, repeat this step until your output matches the example
shown below.
ble
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
fe r a
> alter flashcache all flush
ans
qr01cel01: Flash cache qr01cel01_FLASHCACHE altered successfully
n - t r
qr01cel02: Flash cache qr01cel02_FLASHCACHE altered successfully
a no
as ฺ
qr01cel03: Flash cache qr01cel03_FLASHCACHE altered successfully
18. Drop the Exadata Smart Flash Cache on all of the cells. ) h
c o m uide cellcli -e \
e c hฺ ent G
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03

fot Studsuccessfully dropped


> drop flashcache

i n
qr01cel01: Flash cache qr01cel01_FLASHCACHE

e zy this
qr01cel02: Flash cache qr01cel02_FLASHCACHE successfully dropped
qr01cel03: Flash cache
h i @ se
qr01cel03_FLASHCACHE successfully dropped

b i r
[celladmin@qr01cel01
so to u ~]$

19. Stop cellsrv


( j a son all nofsthee cells.
o h i l i ce
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
S
bir qr01cel01:
> alter cell shutdown services cellsrv

J a s qr01cel01: Stopping CELLSRV services...


qr01cel01: The SHUTDOWN of CELLSRV services was successful.
qr01cel02:
qr01cel02: Stopping CELLSRV services...
qr01cel02: The SHUTDOWN of CELLSRV services was successful.
qr01cel03:
qr01cel03: Stopping CELLSRV services...
qr01cel03: The SHUTDOWN of CELLSRV services was successful.
[celladmin@qr01cel01 ~]$

20. Set the flashCacheMode attribute back to WriteThrough.


[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
> alter cell flashCacheMode = WriteThrough
qr01cel01: Cell qr01cel01 successfully altered
qr01cel02: Cell qr01cel02 successfully altered
qr01cel03: Cell qr01cel03 successfully altered
[celladmin@qr01cel01 ~]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 7
21. Restart cellsrv on all of the cells.
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
> alter cell startup services cellsrv
qr01cel01:
qr01cel01: Starting CELLSRV services...
qr01cel01: The STARTUP of CELLSRV services was successful.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

qr01cel02:
qr01cel02: Starting CELLSRV services...
qr01cel02: The STARTUP of CELLSRV services was successful.
qr01cel03:
qr01cel03: Starting CELLSRV services...
qr01cel03: The STARTUP of CELLSRV services was successful.
[celladmin@qr01cel01 ~]$
22. Re-create Exadata Smart Flash Cache on all the cells.
ble
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \
fe r a
> create flashcache all
ans
qr01cel01: Flash cache qr01cel01_FLASHCACHE successfully created
n - t r
no
qr01cel02: Flash cache qr01cel02_FLASHCACHE successfully created
a
qr01cel03: Flash cache qr01cel03_FLASHCACHE successfully created
[celladmin@qr01cel01 ~]$
) has eฺ
ฺ c G uid
mof the cells.
23. Verify the cell flashCacheMode attribute setting onoall

t e ch dent
[celladmin@qr01cel01 ~]$ dcli -c qr01cel01,qr01cel02,qr01cel03 cellcli -e \

nfo Stu
> list cell attributes flashCacheMode
qr01cel01: WriteThrough y i
z this
qr01cel02: WriteThrough e

o h
qr01cel03: WriteThroughi@ use
b irs e ~]$
[celladmin@qr01cel01
s to
i c e ns session and restart Grid Infrastructure, ASM, and Oracle
ja root terminal
24. Return to (you
oh
Database.
S li
a s bir [root@qr01db01 grid]# crsctl start crs
J CRS-4123: Oracle High Availability Services has been started.
[root@qr01db01 grid]#
25. Wait for a few minutes until your database restarts. Execute the following command to
monitor the status of the database. Proceed to the next practice only after your
database is started on qr01db01.
[root@qr01db01 grid]# srvctl status database -d dbm
Instance dbm1 is running on node qr01db01
Instance dbm2 is not running on node qr01db02
[root@qr01db01 grid]#
26. Exit all of your terminal sessions.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 8
Practice 8-2: Using Exadata Hybrid Columnar Compression
Overview
In this practice, you will examine the performance of Exadata Hybrid Columnar Compression.
You will compare predicted and actual compression ratios using an example dataset. You will
also examine how bulk data loading and query operations are affected using the different
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

compression modes.

Tasks
1. Establish a terminal connection to your database server as the oracle user.
2. Connect to your database with SQL*Plus. Log in as the sales user.
[oracle@qr01db01 ~]$ sqlplus sales/sales

ble
SQL*Plus: Release 11.2.0.3.0 Production...
fe r a
ans
SQL>
n - t r
3. Configure the session to display server output and timing statistics.
a no
SQL> set serveroutput on
) h as ฺ
SQL> set timing on
o m u i de
SQL> c
hฺ ent G
e c
t tud to predict the expected
4. Use dbms_compression.get_compression_ratio
i n f otable S
zy SQL
compression ratio for the MYCUSTOMERS
Columnar Compression modese(use theth
using all the different Exadata Hybrid
is script
h
04.sql if you prefer).oNotei@ that u s
this e
step can take
/home/oracle/labs/lab08-02-
more than 10 minutes to complete.
r
bi se ts o
a s
SQL> declare
2 (j b_cmp number;
n
h i l i c enumber;
i r So 4 r_cmp number;
3 b_ucmp

b
Jas 5
6
r_ucmp number;
cmp_ratio number(6,2);
7 cmp_type varchar2(1024);
8 begin
9 dbms_compression.get_compression_ratio('USERS','SALES',
10 'MYCUSTOMERS',NULL,DBMS_COMPRESSION.COMP_FOR_QUERY_LOW,
11 b_cmp,b_ucmp, r_cmp, r_ucmp, cmp_ratio, cmp_type);
12 dbms_output.put_line('Table: MYCUSTOMERS');
13 dbms_output.put_line('Compression Ratio: '||cmp_ratio);
14 dbms_output.put_line('Compression Type: '|| cmp_type);
15 end;
16 /
Compression Advisor self-check validation successful. select count(*) on both
Uncompressed and EHCC Compressed format = 1000001 rows
Table: MYCUSTOMERS
Compression Ratio: 4.2
Compression Type: "Compress For Query Low"

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 9
PL/SQL procedure successfully completed.

Elapsed: 00:02:49.58
SQL> declare
2 b_cmp number;
3 b_ucmp number;
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

4 r_cmp number;
5 r_ucmp number;
6 cmp_ratio number(6,2);
7 cmp_type varchar2(1024);
8 begin
9 dbms_compression.get_compression_ratio('USERS','SALES',
10 'MYCUSTOMERS',NULL,DBMS_COMPRESSION.COMP_FOR_QUERY_HIGH,
11 b_cmp,b_ucmp, r_cmp, r_ucmp, cmp_ratio, cmp_type);
12 dbms_output.put_line('Table: MYCUSTOMERS');
ble
13 dbms_output.put_line('Compression Ratio: '||cmp_ratio);
fe r a
14 dbms_output.put_line('Compression Type: '|| cmp_type);
ans
15 end;
n - t r
16 /
n o
a
has eฺ
Compression Advisor self-check validation successful. select count(*) on both

)
Uncompressed and EHCC Compressed format = 1000001 rows
Table: MYCUSTOMERS
ฺ c om Guid
Compression Ratio: 7
t e ch dent
nfo Stu
Compression Type: "Compress For Query High"

yi
e
PL/SQL procedure successfully
t h is
z completed.
o h i@ use
s b irs e to
Elapsed: 00:02:47.54

ja ens
SQL> declare
2i ( b_cmp number;
h
So 3 b_ucmplic number;
bir
Jas
4 r_cmp number;
5 r_ucmp number;
6 cmp_ratio number(6,2);
7 cmp_type varchar2(1024);
8 begin
9 dbms_compression.get_compression_ratio('USERS', 'SALES',
10 'MYCUSTOMERS',NULL,DBMS_COMPRESSION.COMP_FOR_ARCHIVE_LOW,
11 b_cmp,b_ucmp, r_cmp, r_ucmp, cmp_ratio, cmp_type);
12 dbms_output.put_line('Table: MYCUSTOMERS');
13 dbms_output.put_line('Compression Ratio: '||cmp_ratio);
14 dbms_output.put_line('Compression Type: '|| cmp_type);
15 end;
16 /
Compression Advisor self-check validation successful. select count(*) on both
Uncompressed and EHCC Compressed format = 1000001 rows
Table: MYCUSTOMERS
Compression Ratio: 9.1
Compression Type: "Compress For Archive Low"

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 10
PL/SQL procedure successfully completed.

Elapsed: 00:02:32.60
SQL> declare
2 b_cmp number;
3 b_ucmp number;
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

4 r_cmp number;
5 r_ucmp number;
6 cmp_ratio number(6,2);
7 cmp_type varchar2(1024);
8 begin
9 dbms_compression.get_compression_ratio('USERS', 'SALES',
10 'MYCUSTOMERS',NULL,DBMS_COMPRESSION.COMP_FOR_ARCHIVE_HIGH,
11 b_cmp,b_ucmp, r_cmp, r_ucmp, cmp_ratio, cmp_type);
12 dbms_output.put_line('Table: MYCUSTOMERS');
ble
13 dbms_output.put_line('Compression Ratio: '||cmp_ratio);
fe r a
14 dbms_output.put_line('Compression Type: '|| cmp_type);
ans
15 end;
n - t r
16 /
a no
has eฺ
Compression Advisor self-check validation successful. select count(*) on both
Uncompressed and EHCC Compressed format = 1000001 rows
)
Table: MYCUSTOMERS
ฺ c om Guid
Compression Ratio: 11.1
t e ch dent
nfo Stu
Compression Type: "Compress For Archive High"

y i
e z completed.
PL/SQL procedure successfully
t h is
o h i@ use
s b irs e to
Elapsed: 00:03:32.62
SQL>
i ( ja Columnar
c e ns Compression achieves its highest levels of compression with
5. Exadatah
othat Hybrid li inserted. Execute the following ALTER SESSION commands to
S
data is direct-path
a s birensure the use of direct-path inserts later in the practice.
J SQL> alter session force parallel query;

Session altered.

Elapsed: 00:00:00.00
SQL> alter session force parallel ddl;

Session altered.

Elapsed: 00:00:00.00
SQL> alter session force parallel dml;

Session altered.

Elapsed: 00:00:00.01
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 11
6. Use the following commands (or execute the SQL script /home/oracle/labs/lab08-
02-06.sql) to create compressed copies of the MYCUSTOMERS table. Notice the relative
difference in the time taken to create each table by using the different compression modes.
SQL> create table mycust_query_low
2 compress for query low
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

3 nologging parallel 4
4 as select * from mycustomers;

Table created.

Elapsed: 00:00:13.53
SQL>
SQL> create table mycust_query_high
ble
2 compress for query high
fe r a
3 nologging parallel 4
ans
4 as select * from mycustomers; n - t r
n o
a
Table created.
) has eฺ
ฺ c om Guid
Elapsed: 00:00:15.13
t e ch dent
nfo Stu
SQL>
yi
ez tlow is
SQL> create table mycust_archive_low
2 compress for@ h
o h i use4
archive

b irs *e from
3 nologging
4 assselect
to mycustomers;
parallel

i ( ja ens
h lic
SoTable created.
b ir
Jas Elapsed: 00:00:22.62
SQL>
SQL> create table mycust_archive_high
2 compress for archive high
3 nologging parallel 4
4 as select * from mycustomers;

Table created.

Elapsed: 00:01:13.98
SQL>
7. Use the following query (or execute the SQL script /home/oracle/labs/lab08-02-
07.sql) to compare the size of the original uncompressed table with the newly created
compressed copies. Calculate the compression ratios achieved using the formula:
Compression Ratio = Uncompressed Size / Compressed Size
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 12
Compare the actual compression ratios achieved with the ones predicted in step 4.
SQL> col segment_name format a30
SQL> select segment_name,sum(bytes)/1024/1024 MB
2 from user_segments
3 where segment_name like 'MYCUST%'
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

4 group by segment_name;

SEGMENT_NAME MB
------------------------------ ----------
MYCUSTOMERS 208
MYCUST_QUERY_LOW 51
MYCUST_QUERY_HIGH 31
MYCUST_ARCHIVE_LOW 27
ble
MYCUST_ARCHIVE_HIGH 19
fe r a
ans
n - t r
no
Elapsed: 00:00:02.74
SQL>
s a
In the next part of the practice, you will compare direct path insert
) haperformance
eฺ for compressed
and uncompressed tables. On each occasion you will perform o m the u
samei dtransaction twice. The
ฺ c G
t result. You should take
h consistent
first time will help to prime the system in order to ensure
particular note of the timings for the second insertt e ccommand.e n
ddata loading
This will help you to determine the
impact of Exadata Hybrid Columnar Compression i n fo on S t u
bulk operations.
z y i
e transactions s
8. As a baseline, execute the following
i @ e thto perform
to load data into the uncompressed
MYCUSTOMERS table. Note
s o h the timestaken
u the second insert.
r
bi /*+APPEND
e o
t */ into mycustomers
a s
SQL> insert
s
2i (jselecte*n from seed_data;
S o h lic
a s bir 200000 rows created.
J
Elapsed: 00:00:10.37
SQL> commit;

Commit complete.

Elapsed: 00:00:00.07
SQL> insert /*+APPEND */ into mycustomers
2 select * from seed_data;

200000 rows created.

Elapsed: 00:00:07.62
SQL> commit;

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 13
Commit complete.

Elapsed: 00:00:00.03
SQL>
9. Execute the same insert transactions against the COMPRESS FOR QUERY LOW copy of the
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

table. Note the time taken to perform the second insert. You may observe that the time for
this insert is better than the uncompressed insert in step 8. In this case, the cost of
performing the compression is offset by the lower number of I/O operations that are
required. This characteristic is one of the reasons why query compression is well suited to
data warehouse environments where large data loads exist.
SQL> insert /*+APPEND */ into mycust_query_low
2 select * from seed_data;

ble
200000 rows created.
fe r a
ans
Elapsed: 00:00:04.36
n - t r
SQL> commit; n o
a
) has eฺ
om Guid
Commit complete.
ฺ c
Elapsed: 00:00:00.04 t e ch dent
SQL> insert /*+APPEND */ininto fo mycust_query_low
S tu
y
ez thi
2 select * from seed_data; s
h i @ s e
o
irscreated.
to u
s
200000 rowsb e
i ( ja ens
h ic
ir SoElapsed: l 00:00:03.07
b
Jas
SQL> commit;

Commit complete.

Elapsed: 00:00:00.03
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 14
10. Execute the same insert transactions against the COMPRESS FOR QUERY HIGH copy of
the table. Note the time taken to perform the second insert and compare it with the previous
results.
SQL> insert /*+APPEND */ into mycust_query_high
2 select * from seed_data;
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

200000 rows created.

Elapsed: 00:00:03.25
SQL> commit;

Commit complete.

ble
Elapsed: 00:00:00.04
fe r a
SQL> insert /*+APPEND */ into mycust_query_high
ans
2 select * from seed_data; n - t r
n o
a
200000 rows created.
) has eฺ
c o m uid
Elapsed: 00:00:03.29
e c h ent G

SQL> commit;
i n fot Stud
e zy this
Commit complete.i@ e
o h u s
s b irs e to
Elapsed:
i ( ja ens
00:00:00.08

So
h
SQL> lic
ir
11. Execute the same insert transactions against the COMPRESS FOR ARCHIVE LOW copy of
b
Jas the table. Note the time taken to perform the second insert. You should observe that the
load times are steadily increasing as more aggressive compression modes are used.
SQL> insert /*+APPEND */ into mycust_archive_low
2 select * from seed_data;

200000 rows created.

Elapsed: 00:00:03.74
SQL> commit;

Commit complete.

Elapsed: 00:00:00.04
SQL> insert /*+APPEND */ into mycust_archive_low
2 select * from seed_data;

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 15
200000 rows created.

Elapsed: 00:00:03.83
SQL> commit;
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Commit complete.

Elapsed: 00:00:00.02
SQL>

12. Execute the insert transaction against the COMPRESS FOR ARCHIVE HIGH copy of the
table. Note the time taken to perform the second insert. This time you should observe a
ble
more substantial cost for the data compression. This is because COMPRESS FOR ARCHIVE
fe r a
HIGH uses a more costly compression algorithm to achieve higher levels of compression.
ans
This extra cost is generally acceptable in archiving situations because the data does not
n - t r
change (or changes very little) after it is loaded. n o
a
SQL> insert /*+APPEND */ into mycust_archive_high
) h as ฺ
2 select * from seed_data;
c o m uide
e c hฺ ent G
200000 rows created.
i n fot Stud
e zy this
Elapsed: 00:00:05.81
SQL> commit; oh
i@ use
s b irs e to
i (
Commit
c e ns
ja complete.
S oh li
ir
Jasb Elapsed: 00:00:00.03
SQL> insert /*+APPEND */ into mycust_archive_high
2 select * from seed_data;

200000 rows created.

Elapsed: 00:00:05.71
SQL> commit;

Commit complete.

Elapsed: 00:00:00.03
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 16
In the final part of the practice, you will compare query performance for compressed and
uncompressed tables.
13. Reconnect to your database as the sales user. This clears the session-level statistics,
which will be used later to compare query performance.
SQL> connect sales/sales
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Connected.
SQL>
14. Execute the following test query against the uncompressed table. Note the time taken to
execute the query.
SQL> select avg(cust_credit_limit) from mycustomers;

AVG(CUST_CREDIT_LIMIT)
----------------------
ble
6176.17987
fe r a
ans
n - t r
no
Elapsed: 00:00:09.07
SQL> a
15. Examine the I/O statistics for the query you just ran (use the
) as scriptฺ
hSQL
/home/oracle/labs/lab08-02-15.sql if you prefer).
c o ideprovide a baseline for
m Thisuwill
later comparison.
e c hฺ ent G
f
SQL> select a.name, b.value/1024/1024
i n ot MBtud
zy tbhis Sand
2 from v$sysstat a, v$mystat
3 where a.statistic# e

o h i@ uread
4 (a.name in ('physical
= b.statistic#
s e total bytes',
5
s b irs 'physical
e to write total bytes',
6
i ( ja ens'cell IO uncompressed bytes')
S oh a.name
7 or lic like 'cell phy%');
a s bir NAME MB
J ---------------------------------------------------------------- ----------
physical read total bytes 291.875
physical write total bytes 0
cell physical IO interconnect bytes 18.3570175
cell physical IO bytes saved during optimized file creation 0
cell physical IO bytes saved during optimized RMAN file restore 0
cell physical IO bytes eligible for predicate offload 291.859375
cell physical IO bytes saved by storage index 0
cell physical IO bytes sent directly to DB node to balance CPU 0
cell physical IO interconnect bytes returned by smart scan 18.3413925
cell IO uncompressed bytes 291.859375

10 rows selected.

Elapsed: 00:00:00.14
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 17
16. Reconnect to your database as the sales user.
SQL> connect sales/sales
Connected.
SQL>
17. Execute the following test query against the COMPRESS FOR QUERY LOW copy of the
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

table. Compare the time taken to execute the query with the query performance observed in
step 14.
SQL> select avg(cust_credit_limit) from mycust_query_low;

AVG(CUST_CREDIT_LIMIT)
----------------------
6176.17987
ble
Elapsed: 00:00:07.06 fe r a
ans
SQL>
n - t r
18. Examine the I/O statistics for the query you just ran (use the SQL script
n o
a
/home/oracle/labs/lab08-02-15.sql if you prefer). Compare the statistics with the
has eฺ
results observed in step 15. Note the substantial decrease in the I/O required to satisfy the
)
query.
ฺ c om Guid
2 from v$sysstat a, v$mystat b tec
SQL> select a.name, b.value/1024/1024 MB h ent
yi nfo Sand
3 where a.statistic# = b.statistic# tud
4 (a.name in ('physicale h is bytes',
z read ttotal
5
o h i@ uwrite
'physical
s e total bytes',
6
s b irslike
'cell o
e t IO uncompressed bytes')

i ( ja ens
7 or a.name 'cell phy%');

oh
S NAME lic
ir
MB

asb
---------------------------------------------------------------- ----------
J physical read total bytes 79.6796875
physical write total bytes 0
cell physical IO interconnect bytes 12.45961
cell physical IO bytes saved during optimized file creation 0
cell physical IO bytes saved during optimized RMAN file restore 0
cell physical IO bytes eligible for predicate offload 72.5625
cell physical IO bytes saved by storage index 0
cell physical IO bytes sent directly to DB node to balance CPU 0
cell physical IO interconnect bytes returned by smart scan 5.34242249
cell IO uncompressed bytes 247.398615

10 rows selected.

Elapsed: 00:00:00.01
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 18
19. Reconnect to your database as the sales user.
SQL> connect sales/sales
Connected.
SQL>
20. Execute the following test query against the COMPRESS FOR QUERY HIGH copy of the
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

table. Compare the time taken to execute the query with the query performance observed
previously.
SQL> select avg(cust_credit_limit) from mycust_query_high;

AVG(CUST_CREDIT_LIMIT)
----------------------
6176.17987
ble
fe r a
Elapsed: 00:00:04.23
ans
SQL>
n - t r
n
21. Examine the I/O statistics for the query you just ran (use the SQL script o
a
/home/oracle/labs/lab08-02-15.sql if you prefer). Compare the statistics with the
has eฺ
results observed previously. Note again the decline in the I/O required to satisfy the query.
)
SQL> select a.name, b.value/1024/1024 MB
ฺ c om Guid
2 from v$sysstat a, v$mystat b
t e chand dent
o bytes',
nftotal
3 where a.statistic# = b.statistic#
i tu
4 (a.name in ('physical y
s S
ez writethtotal
read
i bytes',
5
i @
'physical
e
h IO uuncompressed
s
6
r s o'cell
o
bytes')

a s bi se t
7 or a.name like 'cell phy%');

h i (j icen
o
NAME l MB

b ir S ---------------------------------------------------------------- ----------
Jas
physical read total bytes 52.96875
physical write total bytes 0
cell physical IO interconnect bytes 11.8664856
cell physical IO bytes saved during optimized file creation 0
cell physical IO bytes saved during optimized RMAN file restore 0
cell physical IO bytes eligible for predicate offload 45.984375
cell physical IO bytes saved by storage index 0
cell physical IO bytes sent directly to DB node to balance CPU 0
cell physical IO interconnect bytes returned by smart scan 4.8821106
cell IO uncompressed bytes 248.547052

10 rows selected.

Elapsed: 00:00:00.01
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 19
22. Reconnect to your database as the sales user.
SQL> connect sales/sales
Connected.
SQL>
23. Execute the following test query against the COMPRESS FOR ARCHIVE LOW copy of the
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

table. Compare the time taken to execute the query with the query performance observed in
previously.
SQL> select avg(cust_credit_limit) from mycust_archive_low;

AVG(CUST_CREDIT_LIMIT)
----------------------
6176.17987
ble
fe r a
Elapsed: 00:00:03.50
ans
SQL>
n - t r
n
24. Examine the I/O statistics for the query you just ran (use the SQL script o
a
/home/oracle/labs/lab08-02-15.sql if you prefer). Compare the statistics with the
has eฺ
results observed previously. Note the continued decline in the I/O required to satisfy the
)
query.
c o m uid
SQL> select a.name, b.value/1024/1024 MB
e c h ent G

2 from v$sysstat a, v$mystat b
i n fot Sand tud
y is bytes',
z read ttotal
3 where a.statistic# = b.statistic#
e
i@ uwrite
4 (a.name in ('physical h
e total bytes',
5
o h
'physical s
6
s b irs 'cell
e toIO uncompressed bytes')
7 oraa.name like
i ( j
c e ns 'cell phy%');
oh
S NAME li
ir MB

Jasb ---------------------------------------------------------------- ----------


physical read total bytes 49.3203125
physical write total bytes 0
cell physical IO interconnect bytes 11.5118637
cell physical IO bytes saved during optimized file creation 0
cell physical IO bytes saved during optimized RMAN file restore 0
cell physical IO bytes eligible for predicate offload 42.046875
cell physical IO bytes saved by storage index 0
cell physical IO bytes sent directly to DB node to balance CPU 0
cell physical IO interconnect bytes returned by smart scan 4.23842621
cell IO uncompressed bytes 249.937677

10 rows selected.

Elapsed: 00:00:00.01
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 20
25. Reconnect to your database as the sales user.
SQL> connect sales/sales
Connected.
SQL>
26. Execute the following test query against the COMPRESS FOR ARCHIVE HIGH copy of the
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

table. Compare the time taken to execute the query with the query performance observed
previously. Note that in all cases, the queries against the compressed tables outperformed
the query against the uncompressed table. With compression, you will often observe
improved query performance for scanning queries because less I/O is required.
SQL> select avg(cust_credit_limit) from mycust_archive_high;

AVG(CUST_CREDIT_LIMIT)
----------------------
ble
6176.17987
fe r a
ans
Elapsed: 00:00:03.53 n - t r
SQL> a no
27. Examine the I/O statistics for the query you just ran (use thehSQL
) as scriptฺ
/home/oracle/labs/lab08-02-15.sql if you prefer).
c o m Compare u idethe statistics with the
results previously observed. Note again the decline
e c t Grequired to satisfy the query.
hฺ in ethenI/O
SQL> select a.name, b.value/1024/1024
i n fbot S MB
tud
2 from v$sysstat a, v$mystat
3 where a.statistic# e
y is and
= zb.statistic#
i@ uread t h
e total bytes',
o h
4 (a.name in ('physical s
5
s b irs 'physical
e toIO uncompressed
write total bytes',
6
( j a e n s
'cell bytes')

S oh7i or a.name lic like 'cell phy%');


a s bir NAME MB
J ---------------------------------------------------------------- ----------
physical read total bytes 34.125
physical write total bytes 0
cell physical IO interconnect bytes 9.19754791
cell physical IO bytes saved during optimized file creation 0
cell physical IO bytes saved during optimized RMAN file restore 0
cell physical IO bytes eligible for predicate offload 28.265625
cell physical IO bytes saved by storage index 0
cell physical IO bytes sent directly to DB node to balance CPU 0
cell physical IO interconnect bytes returned by smart scan 3.33817291
cell IO uncompressed bytes 247.281427

10 rows selected.

Elapsed: 00:00:00.01
SQL>
28. Exit your SQL*Plus session.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 21
Practice 8-3: Testing Index Elimination
Overview
In this practice, you make an index invisible so that you can test the effect of removing the index
on your queries without actually dropping the index.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Tasks
1. Establish a terminal connection to your database server.
2. Connect to your database with SQL*Plus. Log in as the sales user.
[oracle@qr01db01 ~]$ sqlplus sales/sales

SQL*Plus: Release 11.2.0.3.0 Production...

ble
SQL>
fe r a
3. Configure your session to display timing statistics and execution plans. Then flush the
ans
database buffer cache to ensure consistent results in the following steps.
n - t r
n o
SQL> set timing on
a
SQL> set autotrace on explain
) has eฺ
SQL> alter system flush buffer_cache;
ฺ c om Guid
t e ch dent
nfo Stu
System altered.
yi
z this
e
Elapsed: 00:00:00.47
o h i@ use
SQL>
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 22
4. Execute the following query. Note that the execution plan uses an index range scan on the
CUSTOMERS_PK index. Note also the time taken to execute the query using the index.
SQL> select avg(cust_credit_limit) from customers
2 where cust_id between 200000 and 320000;

AVG(CUST_CREDIT_LIMIT)
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

----------------------
7682.74014

Elapsed: 00:00:06.43

Execution Plan
----------------------------------------------------------
Plan hash value: 3995619262
bl e
------------------------------------------------------------------------------
fe r a
| Rows | Bytes | n s
tra
| Id | Operation | Name Cost
n -
------------------------------------------------------------------------------
1n|o
| 0 | SELECT STATEMENT | |
|s
a 10 | 9837
| 1 | SORT AGGREGATE |
TABLE ACCESS BY INDEX ROWID| CUSTOMERS) h a 1 |

10 |
| 2 | | e 1171K| 9837
d120K|
o m |ui 120K|
|* 3 | INDEX RANGE SCAN
ฺ c tG
| CUSTOMERS_PK
h
| 260

e c den
------------------------------------------------------------------------------
t
i nf o tu
y s S
@ ez thbyi operation id):
o h i
Predicate Information (identified
u s e
irs e to
---------------------------------------------------

s b
3 -ja
i ( e ns
access("CUST_ID">=200000
c
AND "CUST_ID"<=320000)

oh li
b i r S SQL>
Jas5. Reconfigure your session to disable the automatic output of execution plans.
SQL> set autotrace off
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 23
6. Make the CUSTOMERS_PK index invisible. An invisible index still exists and is maintained by
DML operations, but it is not used by the optimizer for queries.
SQL> alter index customers_pk invisible;

Index altered.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Elapsed: 00:00:00.35
SQL>

7. The index you have just made invisible is associated with a primary key constraint. Use the
following query to check the status of the constraint. Note that even though the index is
invisible, the associated constraint is still enabled.
SQL> select status from user_constraints
ble
2 where constraint_name='CUSTOMERS_PK';
fe r a
ans
STATUS
n - t r
--------
n o
a
has eฺ
ENABLED

)
Elapsed: 00:00:00.64
ฺ c om Guid
SQL>
e c h ent
t ud plans.
fo showSexecution
8. Reconfigure your session to automatically
t
yi n
SQL> set autotrace onz explain
e t h is
SQL>
o h i@ use
s b irs e to
i ( ja ens
S oh lic
a s bir
J

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 24
9. Re-execute the query from step 4. Notice that an Exadata Smart Scan is used rather than
an index range scan.
SQL> select avg(cust_credit_limit) from customers
2 where cust_id between 200000 and 320000;

AVG(CUST_CREDIT_LIMIT)
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

----------------------
7682.74014

Elapsed: 00:00:19.45

Execution Plan
----------------------------------------------------------
Plan hash value: 296924608
bl e
------------------------------------------------------------------------------
fe r a
| Rows | Bytes | Cost n s
tra
| Id | Operation | Name (%CPU)|

n -
------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
a n10o | 12093 (1)|
1s |
| 1 | SORT AGGREGATE | |
h a
TABLE ACCESS STORAGE FULL| CUSTOMERS |) 120K| e ฺ
10 | |
|* 2 |
o m u i d 1171K| 12093 (1)|
c h ฺc nt G
------------------------------------------------------------------------------

f o te tude
z
Predicate Information (identifiedyin bys operation
S id):
e h i
h i @ se t
---------------------------------------------------

b i rso e to u
2 - storage("CUST_ID"<=320000 AND "CUST_ID">=200000)

( j a s n s
filter("CUST_ID"<=320000 AND "CUST_ID">=200000)
i c e
oh li
b i r S SQL>
JasCompare the time taken to execute the query with and without the index. Note that in some
cases, Exadata Smart Scan may deliver better query performance than using an index.
However, this may not always be the case. Even in cases where an index delivers better query
performance you might choose to remove it if you determine that the un-indexed query
performance is acceptable and the index is otherwise unnecessary. Removing unnecessary
indexes saves space and improves DML performance by eliminating the maintenance
operations associated with the index. If you decide not to remove the index, you can quickly and
easily make it visible.
10. Make the index visible again.
SQL> alter index customers_pk visible;

Index altered.

Elapsed: 00:00:00.04
SQL>
11. Exit your SQL*Plus session.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 25
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Recommendations for Optimizing Database Performance


Chapter 8 - Page 26
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
Practices )for s
haLesson
m i d eฺ 9: Using
Smart ฺ coScan G u
e h
c 9 den t
o t
Chapter
y i nf S tu
ez thi s
h i @ s e
o
irs e to u
s b
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 1
Practices for Lesson 9
Practices Overview
In these practices, you will exercise Exadata Smart Scan and examine various statistics and
wait events to determine what is occurring.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 2
Practice 9-1: Monitoring Exadata Smart Scan
Overview
In this practice, you will examine various Exadata Smart Scan statistics and measures which
can be observed within Oracle Database.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Tasks
1. Establish a terminal connection to qr01db01 as the oracle user.
2. Connect to your database with SQL*Plus. Login as the sales user.
[oracle@qr01db01 ~]$ sqlplus sales/sales

SQL*Plus: Release 11.2.0.3.0 Production...

ble
SQL>
fe r a
3. Flush the buffer cache to ensure a consistent starting point for this practice, and reconnect
ans
to the database to reset the session statistics. Then, configure your session to display
n - t r
query execution plans and statement timings. n o
a
SQL> alter system flush buffer_cache;
) has eฺ
ฺ c om Guid
ch dent
System altered.
t e
SQL> connect sales/sales yi nfo Stu
e z this
Connected.
o h i@onuexplain
s e
irs eonto
SQL> set autotrace
SQL> set s btiming
i ( ja ens
SQL>
So
h lic
b ir
Jas

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 3
4. Execute the following query. You can identify whether Smart Scan is possible by examining
the query execution plan. Smart Scan is indicated for this query by the TABLE ACCESS
STORAGE FULL operation. You can also see that the WHERE clause predicate
(occupation = 'Farming') can be evaluated by Exadata.
SQL> select count(*) from cust_info where occupation = 'Farming';
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

COUNT(*)
----------
13845

Elapsed: 00:00:03.58

Execution Plan
----------------------------------------------------------
ble
Plan hash value: 1273666552
fe r a
ans
------------------------------------------------------------------------------
n - t r
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|
n o
a
------------------------------------------------------------------------------
| 0 | SELECT STATEMENT |
)
|
has eฺ
1 | 7 | 4176 (1)|

om Guid
| 1 | SORT AGGREGATE | | 1 | 7 | |
|* 2 |
ฺ c
TABLE ACCESS STORAGE FULL| CUST_INFO | 115K| 788K| 4176 (1)|

e ch dent
------------------------------------------------------------------------------
t
yi nfo Stu
e z thbyis operation id):
o h i@ use
Predicate Information (identified

irs e to
---------------------------------------------------

s b
2 (-ja
i e ns
storage("OCCUPATION"='Farming')
c
h filter("OCCUPATION"='Farming')
li
ir So
b
Jas5.
SQL>
Reconfigure your session to disable the automatic output of execution plans.
SQL> set autotrace off
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 4
6. Execute the following query (or execute the SQL script /home/oracle/labs/lab09-
01-06.sql) and examine the statistics for the current session. Since the query in step 4 is
the only query that has been executed during this session we can safely assume that the
statistics relate to that query. The statistics show that approximately 120 MB of I/O was
performed to scan the cust_info table (physical read total bytes), and that
almost all of the I/O was eligible for Smart Scan (cell physical IO bytes eligible
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

for predicate offload). The statistics further show that Smart Scan returned
approximately 200 KB of data back to the database server (cell physical IO
interconnect bytes returned by smart scan).
SQL> SELECT s.name, m.value/1024/1024 MB FROM V$SYSSTAT s, V$MYSTAT m
2 WHERE s.statistic# = m.statistic# AND
3 (s.name LIKE 'physical%total bytes' OR s.name LIKE 'cell phys%'
4 OR s.name LIKE 'cell IO%');

ble
NAME MB
fe r a
----------------------------------------------------------------
s
----------
an
physical read total bytes
n - t r
119.875
physical write total bytes
n o 0
cell physical IO interconnect bytes
a .555992126

) has eฺ
cell physical IO bytes saved during optimized file creation 0

om Guid
cell physical IO bytes saved during optimized RMAN file restore 0

ฺ c
cell physical IO bytes eligible for predicate offload 119.523438

t e ch dent
cell physical IO bytes saved by storage index 0

yi nfo Stu
cell physical IO bytes sent directly to DB node to balance CPU 0

e z this
cell physical IO interconnect bytes returned by smart scan .204429626

o h i@ use
cell IO uncompressed bytes 119.523438

s b irs e to
10 rows selected.

i ( ja ens
h
oElapsed:
S SQL> lic
00:00:00.11
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 5
7. Execute the following query (or execute the SQL script /home/oracle/labs/lab09-
01-07.sql) and examine the cell wait events associated with the current session. Note
that the amount of time associated with the cell smart table scan wait event
accounts for most of the execution time observed in step 4. This is normal for a query using
Smart Scan. You may also see some other wait events that relate to other activities in the
session, such as executing the statistics queries in this step and the prior step.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

SQL> SELECT DISTINCT event, total_waits,


2 time_waited/100 wait_secs, average_wait/100 avg_wait_secs
3 FROM V$SESSION_EVENT e, V$MYSTAT s
4 WHERE event LIKE 'cell%' AND e.sid = s.sid;

EVENT TOTAL_WAITS
---------------------------------------------------------------- -----------
WAIT_SECS AVG_WAIT_SECS
ble
---------- -------------
cell smart table scan 37
fe r a
ans
3.05 .0824

n - t r
cell single block physical read
a no 75

has eฺ
1.09 .0145
)
ฺ c om Guid
Elapsed: 00:00:00.10
e c h ent
SQL> t
fo Stud
y i n
Examining the statistics and wait events
e z associated
t h isSmart
with the query executed at step 4 indicates
that the query did make efficient
h i @ se
use of Exadata Scan just as the query execution plan
suggested. In the next part
i r s o to u
of this practice you will consider a scenario where the query

something more.( j a sb the


execution plan indicates
n suse
e of Smart Scan but the statistics and wait events suggest
8. Leave o i current
hyour l iceSQL session active and establish a second terminal connection to
i S
rqr01db01 as the oracle user.
a s b
J 9. In the second terminal, connect to your database with SQL*Plus. Log in as the sales user
and set the SQL prompt to UPDATE> so that you can easily distinguish this session from
your other SQL session.
[oracle@qr01db01 ~]$ sqlplus sales/sales

SQL*Plus: Release 11.2.0.3.0 Production...

SQL> set sqlprompt "UPDT> "


UPDT>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 6
10. In the second terminal, execute the following command to update a substantial number of
customer records. Following the update, flush the buffer cache. This simulates a long
running update transaction where the updated blocks (and associated rollback segment
blocks) have been aged out of the buffer cache. Leave the transaction in this terminal
window uncommitted for now. Do not proceed to the next step until the update
command completes and the buffer cache is flushed.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

UPDT> update cust_info set


2 affinity_card = 0
3 where occupation = 'Farming';

13845 rows updated.

UPDT> alter system flush buffer_cache;


ble
fe r a
System altered.
ans
n - t r
UPDT>
a no
11. Switch back to your first SQL session, leaving the second terminal
h a s session in the
background for now.
m ) i d eฺ
c
12. Back in the first SQL session, reconnect to the database
ฺ G u a fresh database
o to establish
session as the sales user.
e h
c den t
SQL> connect sales/salesinf o t tu
y s S
Connected.
@ ez thi
SQL>
o h i u s e
b
13. Configure your newly
s irs created
e tosession to display query execution plans.
i
SQL> ( a autotrace
jset
c e ns on explain
h
oSQL> li
S
a s bir
J

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 7
14. Re-execute the query from step 4. Notice again that the query execution plan indicates the
use of Smart Scan. Notice also that the execution time increases substantially compared
with step 4.
SQL> select count(*) from cust_info where occupation = 'Farming';

COUNT(*)
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

----------
13845

Elapsed: 00:00:06.05

Execution Plan
----------------------------------------------------------
Plan hash value: 1273666552

bl e
------------------------------------------------------------------------------
fe r a
| Rows | Bytes | Cost n s
tra
| Id | Operation | Name (%CPU)|

n -
------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
a no7 | 4176 (1)|
| 1 | SORT AGGREGATE | |
a1s| 788K|
TABLE ACCESS STORAGE FULL| CUST_INFO |) h115K| ฺ
7 | |

m uide
|* 2 | 4176 (1)|

c o
------------------------------------------------------------------------------

e c hฺ ent G
f o t tud
i n S
e zy thbyis operation
Predicate Information (identified id):

o h i@ use
---------------------------------------------------

b irs e to
2 - storage("OCCUPATION"='Farming')
s
i ( c e ns
jafilter("OCCUPATION"='Farming')
oh li
b i r S SQL>
Jas15. Reconfigure your session to disable the automatic output of execution plans.
SQL> set autotrace off
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 8
16. Execute the following query (or execute the SQL script /home/oracle/labs/lab09-
01-06.sql) and examine the statistics for the current session. Compare the output with
the observations at step 6. This time the same amount of I/O is eligible for Smart Scan
(cell physical IO bytes eligible for predicate offload); however,
substantially more data was transported to the database server (cell physical IO
interconnect bytes returned by smart scan).
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

SQL> SELECT s.name, m.value/1024/1024 MB FROM V$SYSSTAT s, V$MYSTAT m


2 WHERE s.statistic# = m.statistic# AND
3 (s.name LIKE 'physical%total bytes' OR s.name LIKE 'cell phys%'
4 OR s.name LIKE 'cell IO%');

NAME MB
---------------------------------------------------------------- ----------
physical read total bytes 121.90625
ble
physical write total bytes 0
fe r a
cell physical IO interconnect bytes
s
73.2464371
an
cell physical IO bytes saved during optimized file creation
n - t r 0
cell physical IO bytes saved during optimized RMAN file restore
n o 0

a
cell physical IO bytes eligible for predicate offload 119.523438

) has eฺ
cell physical IO bytes saved by storage index 0

om Guid
cell physical IO bytes sent directly to DB node to balance CPU 0

ฺ c
cell physical IO interconnect bytes returned by smart scan 70.8636246
cell IO uncompressed bytes
t e ch dent 119.523438

yi nfo Stu
10 rows selected.
e z this
o h i@ use
irs e to
Elapsed: 00:00:00.00
SQL>
s b
i ( ja ens
So
h lic
bir
Jas

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 9
17. Execute the following query (or execute the SQL script /home/oracle/labs/lab09-
01-07.sql) and examine the cell wait events associated with the current session.
Compare the output with the observations at step 7. Notice that this time a significant
amount of time is associated with the cell single block physical read wait event.
This is because the uncommitted update transaction has forced a substantial number of
reads to be transferred to the traditional buffer cache read-consistency path and since the
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

required blocks were not in the buffer cache a large number of single block physical reads
were required. Notice also the amount of time associated with the different wait events and
how they correlate with the overall query execution time. Clearly, the efficiency and
performance of Smart Scan were severely compromised by the pending transaction.
SQL> SELECT DISTINCT event, total_waits,
2 time_waited/100 wait_secs, average_wait/100 avg_wait_secs
3 FROM V$SESSION_EVENT e, V$MYSTAT s
4 WHERE event LIKE 'cell%' AND e.sid = s.sid;

ble
TOTAL_WAITSra
EVENT
n s fe
tra
---------------------------------------------------------------- -----------
WAIT_SECS AVG_WAIT_SECS
o n -
---------- -------------
a n
cell smart table scan
h a s 65

) ฺ
ide
1.54 .0236

c o m u
cell single block physical read
e c hฺ ent G 305
t
nfo Stud
4.34 .0142
y i
z this
e
Elapsed: 00:00:00.11
o h i@ use
SQL>
s b irs e to
a of this
In the final(jpart
e s
npractice, you will consider another scenario where the query
h
executioni
o different.
plan l i c
indicates the use of Smart Scan but the statistics and wait events suggest
ir S
something
b
Jas18. Switch back to the second SQL session which contains the update transaction.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 10
19. Commit the current transaction and then execute another update command to update a
substantial number of customer records. This time, do not flush the buffer cache after the
update command. Like you did in step 10, leave the transaction in this terminal
window uncommitted for now. Do not proceed to the next step until the update
command completes.
UPDT> commit;
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Commit complete.

UPDT> update cust_info set


2 affinity_card = 1
3 where occupation = 'Farming';

ble
13845 rows updated.
fe r a
ans
UPDT>
n - t r
n
20. Back in the first SQL session, reconnect to the database to establish a fresh database o
a
has eฺ
session as the sales user.
)
om Guid
SQL> connect sales/sales
Connected. ฺ c
SQL> t e ch dent
21. Configure your newly created sessionyi ntofodisplaySquery
tu execution plans.
ezexplainthi s
SQL> set autotrace
h i @ on
e
SQL>
i r s o t o us
( j a sb nse
o hi l ice
b i rS
Jas

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 11
22. Re-execute the query from step 4. Notice again that the query execution plan indicates the
use of Smart Scan. Notice also the execution time difference compared with the previous
runs.
SQL> select count(*) from cust_info where occupation = 'Farming';

COUNT(*)
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

----------
13845

Elapsed: 00:00:00.35

Execution Plan
----------------------------------------------------------
Plan hash value: 1273666552

bl e
------------------------------------------------------------------------------
fe r a
| Rows | Bytes | Cost n s
tra
| Id | Operation | Name (%CPU)|

n -
------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
a no7 | 4176 (1)|
| 1 | SORT AGGREGATE | |
a1s| 788K|
TABLE ACCESS STORAGE FULL| CUST_INFO |) h115K| ฺ
7 | |

m uide
|* 2 | 4176 (1)|

c o
------------------------------------------------------------------------------

e c hฺ ent G
f o t tud
i n S
e zy thbyis operation
Predicate Information (identified id):

o h i@ use
---------------------------------------------------

b irs e to
2 - storage("OCCUPATION"='Farming')
s
i ( c e ns
jafilter("OCCUPATION"='Farming')
oh li
b i r S SQL>
Jas23. Reconfigure your session to disable the automatic output of execution plans.
SQL> set autotrace off
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 12
24. Execute the following query (or execute the SQL script /home/oracle/labs/lab09-
01-06.sql) and examine the statistics for the current session. Now the statistics relate to
the query at step 22. Notice that all the statistics are zero (or very near to zero).
SQL> SELECT s.name, m.value/1024/1024 MB FROM V$SYSSTAT s, V$MYSTAT m
2 WHERE s.statistic# = m.statistic# AND
3 (s.name LIKE 'physical%total bytes' OR s.name LIKE 'cell phys%'
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

4 OR s.name LIKE 'cell IO%');

NAME MB
---------------------------------------------------------------- ----------
physical read total bytes 0
physical write total bytes 0
cell physical IO interconnect bytes 0
cell physical IO bytes saved during optimized file creation 0
ble
cell physical IO bytes saved during optimized RMAN file restore 0
fe r a
cell physical IO bytes eligible for predicate offload 0

ans
cell physical IO bytes saved by storage index
cell physical IO bytes sent directly to DB node to balance CPU
0
0 n - t r
cell physical IO interconnect bytes returned by smart scan
a no
0

has eฺ
cell IO uncompressed bytes 0
)
10 rows selected.
ฺ c om Guid
t e ch dent
Elapsed: 00:00:00.00
y i nfo Stu
SQL>
e z this
25. Execute the following query
o h i@(or execute
u s e the SQL script /home/oracle/labs/lab09-
01-07.sql) and examine
b i s
rtime the
t o cell wait events associated with the current session. This
time the amount
j a s realized
of
n s e
waiting for the cells is zero (or very close to zero). This time, the
database
h i (kernel
i c e that the query could be satisfied using blocks in the buffer cache.
l
So though the query plan indicted the use of Smart Scan, the database kernel used
So even
b i r the buffer cache at runtime and avoided the need for I/O against the cells.
Jas SQL> SELECT DISTINCT event, total_waits,
2 time_waited/100 wait_secs, average_wait/100 avg_wait_secs
3 FROM V$SESSION_EVENT e, V$MYSTAT s
4 WHERE event LIKE 'cell%' AND e.sid = s.sid;

no rows selected

Elapsed: 00:00:00.08
SQL>
In this practice you have seen a variety of scenarios where the optimizer indicated the use
of Smart Scan. However you have also seen that depending on the situation, the
performance of Smart Scan may be impacted by other concurrent transactions, or Smart
Scan may be skipped, partially or completely, if the database kernel can make use of
information in the buffer cache to avoid I/O operations.
26. Switch back to the second SQL session which contains the update transaction.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 13
27. Commit the transaction.
UPDATE> commit;

Commit complete.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

UPDATE>
28. Exit all your terminal sessions.

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 14
Practice 9-2: Monitoring Cell Wait Events for Parallel Query
Overview
In this practice, you consider strategies for monitoring cell wait events when parallel query is
used.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

Tasks
1. Establish a terminal connection to qr01db01 as the oracle user.
2. Connect to your database with SQL*Plus. Log in as the sales user.
[oracle@qr01db01 ~]$ sqlplus sales/sales

SQL*Plus: Release 11.2.0.3.0 Production...

ble
SQL>
fe r a
3. Configure your session to display query execution plans and statement timings.
ans
n - t r
no
SQL> set autotrace on explain
SQL> set timing on a
SQL>
) has eฺ
4. Typically Exadata Smart Scan is used in conjunctionowith d Configure your
m paralleluiquery.
ฺ c G
session to force the use of parallel query.
t e ch dent
SQL> alter session force parallel
y i nfo Stuquery parallel 2;
e z this
Session altered.i@ e
o h u s
s b irs e to
( ja 00:00:00.00
Elapsed:
e n s
h i
oSQL> li c
S
a s bir
J

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 15
5. Execute the following query. The query execution plan indicates the use of parallel query
and Exadata Smart Scan.
SQL> select count(*) from cust_info where occupation = 'Farming';

COUNT(*)
----------
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

13845

Elapsed: 00:01:19.23

Execution Plan
----------------------------------------------------------
Plan hash value: 3555626242

------------------------------------------------------------------------------ ble
| Id | Operation | Name | Rows | Bytes | Cost
fe r a
------------------------------------------------------------------------------
ans
| 0 | SELECT STATEMENT | | 1 | 7 | 2317
n - t r
| 1 | SORT AGGREGATE | | n o
a 1 | 7 |

has eฺ
| 2 | PX COORDINATOR | | | |
)
om Guid
| 3 | PX SEND QC (RANDOM) | :TQ10000 | 1 | 7 |
| 4 | SORT AGGREGATE
ฺ|
c | 1 | 7 |
| 5 | PX BLOCK ITERATOR
t e ch dent
| | 115K| 788K| 2317

nfo Stu
|* 6 | TABLE ACCESS STORAGE FULL| CUST_INFO | 115K| 788K| 2317
yi
z this
------------------------------------------------------------------------------
e
o h i@ use
b irs e t(identified
Predicate Information
s
o by operation id):
a
(j icen s
---------------------------------------------------
h i l
ir So 6 - storage("OCCUPATION"='Farming')
b
Jas
filter("OCCUPATION"='Farming')

SQL>
6. Reconfigure your session to disable the automatic output of execution plans.
SQL> set autotrace off
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 16
7. Execute the following query (or execute the SQL script /home/oracle/labs/lab09-
02-07.sql) and examine the statistics for the current session. The statistics confirm the
use of Smart Scan for the query in step 5.
SQL> SELECT s.name, m.value/1024/1024 MB FROM V$SYSSTAT s, V$MYSTAT m
2 WHERE s.statistic# = m.statistic# AND
3 (s.name LIKE 'physical%total bytes' OR s.name LIKE 'cell phys%'
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

4 OR s.name LIKE 'cell IO%');

NAME MB
---------------------------------------------------------------- ----------
physical read total bytes 119.53125
physical write total bytes 0
cell physical IO interconnect bytes .219009399
cell physical IO bytes saved during optimized file creation 0
ble
cell physical IO bytes saved during optimized RMAN file restore 0
fe r a
cell physical IO bytes eligible for predicate offload
s
119.523438

an
cell physical IO bytes saved by storage index
cell physical IO bytes sent directly to DB node to balance CPU n - t r 0
0
cell physical IO interconnect bytes returned by smart scan
a no .211196899

has eฺ
cell IO uncompressed bytes 119.523438
)
10 rows selected.
ฺ c om Guid
t e ch dent
Elapsed: 00:00:00.03
y i nfo Stu
SQL>
e z this
8. Execute the following query
o h i@(or execute
u s e the SQL script /home/oracle/labs/lab09-
02-08.sql) and examine
b s
irfew the
towaits. events associated with the current session. Notice
cell wait
that there are s
very (if e
any)
i ( ja ens event, total_waits,
S
SQL>
lic
oh2 time_waited/100
SELECT DISTINCT

b i r wait_secs, average_wait/100 avg_wait_secs

Jas
3 FROM V$SESSION_EVENT e, V$MYSTAT s
4 WHERE event LIKE 'cell%' AND e.sid = s.sid;

no rows selected

Elapsed: 00:00:00.06
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 17
Based on the statistics in step 7, you might reasonably expect to see wait events for cell
smart table scan in step 8. What happened? Because parallel query was used, the query
I/O was performed by parallel server processes. The associated wait events are connected to
the parallel server sessions, not the current session. Note that this behavior is symptomatic of
parallel query and is not Exadata-specific. So when parallel query is used, the wait events must
be observed differently. The rest of the practice shows two alternative strategies for observing
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

the wait events.


9. Execute the following query (or execute the SQL script /home/oracle/labs/lab09-
02-09.sql) to display the cell wait events across the entire system.
SQL> select event, total_waits,
2 time_waited/100 wait_secs, average_wait/100 avg_wait_secs
3 from v$system_event where event like 'cell%';

EVENT TOTAL_WAITS
ble
---------------------------------------------------------------- -----------
fe r a
WAIT_SECS AVG_WAIT_SECS
ans
---------- -------------
n - t r
cell smart table scan
n
9277
o
550.44 .0593
a
) has eฺ
om Guid
cell single block physical read 6099
216.74 .0355
ฺ c
t e ch dent
cell multiblock physical read fo
.0354 zyi
n S tu 1826
64.57
e thi s
i @
h physical e
s
cell list of blocks
r o o us read 37
12.37bi t
( j a s nse .3342

o hi l ice
ir S Elapsed: 00:00:00.24
b
Jas SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 18
10. Re-execute the parallel query.
SQL> select count(*) from cust_info where occupation =
'Farming';
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

COUNT(*)
----------
13845

Elapsed: 00:00:02.91
SQL>
11. Re-execute the following query (or execute the SQL script /home/oracle/labs/lab09-
02-09.sql) to again display the cell wait events across the entire system. Compare the ble
output with the output from step 9. The differences are the cell wait events associated with
fe r a
the query at step 10.
ans
SQL> select event, total_waits,
n - t r
a
2 time_waited/100 wait_secs, average_wait/100 avg_wait_secsno
has eฺ
3 from v$system_event where event like 'cell%';
)
EVENT
ฺ c om Guid TOTAL_WAITS

e c h n t
----------------------------------------------------------------
e -----------
t
WAIT_SECS AVG_WAIT_SECS
yi nfo Stud
---------- -------------
cell smart table scan e
z this
h i @ se 9404

rso e to u
556.14 .0591

b i
s block
( j a
cell single
i c e nsphysical read 6099

S oh 216.74 li .0355

ir
Jasb cell multiblock physical read
64.57 .0354
1826

cell list of blocks physical read 37


12.37 .3342

Elapsed: 00:00:00.00
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 19
Using system-level wait event statistics is a simple way to monitor parallel query wait events as
long as you are the only user of the system and you do not wish to monitor concurrent
operations. Often this is not the case. The final part of this practice shows another method
which can be used to isolate the wait events associated with a specific parallel query operation
regardless of concurrency.
12. Execute the following query to determine the default trace file for the current session. Take
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

note of the directory path since that location will also be the default location for other trace
files.
SQL> select value from v$diag_info
2 where name = 'Default Trace File';

VALUE
----------------------------------------------------------------
/u01/app/oracle/diag/rdbms/dbm/dbm1/trace/dbm1_ora_5387.trc ble
fe r a
ans
Elapsed: 00:00:00.40
n - t r
SQL>
n o
a
13. Use the dbms_session.set_identifier procedure to set asclient identifier (PQ1) for
hainformation
the current session. The client identifier will help to locate)trace eฺin behalf
associated with
the current session and any parallel query sessions o m
that u
perform i d
work of the
ฺ c G
current session.
t e ch dent
nfo Stu
SQL> exec dbms_session.set_identifier(client_id=>'PQ1')
y i
e z completed.
t h is
o h i@ use
PL/SQL procedure successfully

b irs e to
Elapsed: 00:00:00.17
s
SQL> ja
i ( c e ns
h dbms_session.client_id_trace_enable
14. Useothe li procedure to start recording trace
S
a s birthat
information for the PQ1 client identifier. Notice that waits=>true is specified to ensure
wait information is recorded in the trace.
J SQL> exec dbms_monitor.client_id_trace_enable(client_id=>'PQ1', waits=>true,
binds=>false)

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.20
SQL>

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 20
15. Re-execute the parallel query.
SQL> select count(*) from cust_info where occupation =
'Farming';

COUNT(*)
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

----------
13845

Elapsed: 00:00:03.20
SQL>
16. Stop the trace gathering started in step 14 and exit SQL*Plus.
SQL> exec dbms_monitor.client_id_trace_disable(client_id=>'PQ1')

ble
PL/SQL procedure successfully completed.
fe r a
ans
Elapsed: 00:00:00.00
n - t r
SQL> exit
a no
has eฺ
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0...
[oracle@qr01db01 ~]$
)
c om inGstep
17. Change directories to the location of the trace file observed
ฺ uid12.
t e ch dent
[oracle@qr01db01 ~]$ cd /u01/app/oracle/diag/rdbms/dbm/dbm1/trace
[oracle@qr01db01 trace]$
y i nfo Stu
18. Search for trace files containing e CLIENT is
z thID:(PQ1). The resulting output will include the
h
trace file for the client SQLi @
sessions e
that was listed in the query output from step 12. It will
o
rs associated
also include traceifiles u
to with any parallel query servers that performed work for
the same client.s b e
i ( ja enstrace]$ grep "CLIENT ID:(PQ1)" *
S
h lic
[oracle@qr01db01
odbm1_ora_5387.trc:***
b i r dbm1_p000_28409.trc:***CLIENT ID:(PQ1) 2013-07-18 02:40:33.367

Jas
CLIENT ID:(PQ1) 2013-07-18 02:40:45.726
dbm1_p001_28413.trc:*** CLIENT ID:(PQ1) 2013-07-18 02:40:45.727
[oracle@qr01db01 trace]$

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 21
19. Examine the trace files listed in step 18. The parallel query server trace files will display the
cell smart table scan events associated with the parallel query.
[oracle@qr01db01 trace]$ grep cell dbm1_p001_28413.trc
...
WAIT #140046319890584: nam='cell smart table scan' ela= 60022
cellhash#=1662637845 p2=0 p3=0 obj#=77120 tim=1374129647458319
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

WAIT #140046319890584: nam='cell smart table scan' ela= 330


cellhash#=3713325327 p2=0 p3=0 obj#=77120 tim=1374129647459253
WAIT #140046319890584: nam='cell smart table scan' ela= 171120
cellhash#=3713325327 p2=0 p3=0 obj#=77120 tim=1374129647630414
WAIT #140046319890584: nam='cell smart table scan' ela= 1004
cellhash#=1662637845 p2=0 p3=0 obj#=77120 tim=1374129647632049
WAIT #140046319890584: nam='cell smart table scan' ela= 3 cellhash#=2749642338
p2=0 p3=0 obj#=77120 tim=1374129647632089
WAIT #140046319890584: nam='cell smart table scan' ela= 131068
cellhash#=2749642338 p2=0 p3=0 obj#=77120 tim=1374129647763208
ble
WAIT #140046319890584: nam='cell smart table scan' ela= 15
fe r a
cellhash#=1662637845 p2=0 p3=0 obj#=77120 tim=1374129647763363
ans
WAIT #140046319890584: nam='cell smart table scan' ela= 98818
n - t r
cellhash#=1662637845 p2=0 p3=0 obj#=77120 tim=1374129647862204
n o
a
WAIT #140046319890584: nam='cell smart table scan' ela= 288

has eฺ
cellhash#=2749642338 p2=0 p3=0 obj#=77120 tim=1374129647863147
)
om Guid
WAIT #140046319890584: nam='cell smart table scan' ela= 246864
cellhash#=2749642338 p2=0 p3=0 obj#=77120 tim=1374129648110052
ฺ c
t e ch dent
WAIT #140046319890584: nam='cell smart table scan' ela= 349

nfo Stu
cellhash#=3713325327 p2=0 p3=0 obj#=77120 tim=1374129648111076

yi
WAIT #140046319890584: nam='cell smart table scan' ela= 164544
z this
e
cellhash#=3713325327 p2=0 p3=0 obj#=77120 tim=1374129648275729

o h i@ use
WAIT #140046319890584: nam='cell smart table scan' ela= 395

irs e to
cellhash#=1662637845 p2=0 p3=0 obj#=77120 tim=1374129648276811

s b
WAIT #140046319890584: nam='cell smart table scan' ela= 3 cellhash#=2749642338
( ja ens
p2=0 p3=0 obj#=77120 tim=1374129648276841
i
S oh lic
WAIT #140046319890584: nam='cell smart table scan' ela= 301395

ir
cellhash#=2749642338 p2=0 p3=0 obj#=77120 tim=1374129648578285

asb
WAIT #140046319890584: nam='cell smart table scan' ela= 29
J cellhash#=2749642338 p2=0 p3=0 obj#=77120 tim=1374129648578539
WAIT #140046319890584: nam='cell smart table scan' ela= 25091
cellhash#=2749642338 p2=0 p3=0 obj#=77120 tim=1374129648603655
WAIT #140046319890584: nam='cell smart table scan' ela= 332
cellhash#=3713325327 p2=0 p3=0 obj#=77120 tim=1374129648604616
WAIT #140046319890584: nam='cell smart table scan' ela= 86345
cellhash#=3713325327 p2=0 p3=0 obj#=77120 tim=1374129648691003
WAIT #0: nam='cell smart table scan' ela= 1030 cellhash#=2749642338 p2=0 p3=0
obj#=77120 tim=1374129648770459
WAIT #0: nam='cell smart table scan' ela= 459 cellhash#=3713325327 p2=0 p3=0
obj#=77120 tim=1374129648770946
WAIT #0: nam='cell smart table scan' ela= 550 cellhash#=1662637845 p2=0 p3=0
obj#=77120 tim=1374129648771519
[oracle@qr01db01 trace]$
20. Exit your terminal session.

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Using Smart Scan


Chapter 9 - Page 22
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
a no
Practices )for s
haLesson
m i d eฺ 10:
Consolidation
ฺ co t GuOptions and
h
c den
Recommendations
e
o t
y i nf Chapter
S tu10
ez thi s
h i @ s e
o
irs e to u
s b
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Consolidation Options and Recommendations


Chapter 10 - Page 1
Practices for Lesson 10
Practices Overview
There is no practice for Lesson 10.
Unauthorized reproduction or distribution prohibitedฺ Copyright© 2014, Oracle and/or its affiliatesฺ

ble
fe r a
ans
n - t r
n o
a
) has eฺ
ฺ c om Guid
t e ch dent
yi nfo Stu
e z this
o h i@ use
s b irs e to
i ( ja ens
S oh lic
ir
Jasb

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Consolidation Options and Recommendations


Chapter 10 - Page 2

You might also like