RDS and Oracle 10g RAC Update

Paul Tsien, Oracle

Agenda
Oracle Database10g RAC What is RDS (Reliable Datagram Sockets)? Open Source RDS for Linux Beta Customer Experience

Oracle Database 10g RAC

Oracle Database 10g Real Application Clusters (RAC) provides the ability to build an application platform from multiple systems that are clustered together Allows applications to become
Highly scaleable Highly available

Oracle Instance

Oracle Instance

Oracle Instance

Shared Database

Chosen to avoid a single node failure, causing application downtime

Eliminates a node as single point of failure
3

Oracle Database 10g RAC

Worlds best Scalability with Cache Fusion
Users

Cache Fusion
he Shared Cac

Cache-to-cache data shipping Scales off-the-shelf applications with no changes

Worlds best Availability with FastStart Fault Recovery

Node failure is transparent to applications Recovers from node failure in 17 seconds - workload independent Pre-warmed cache speeds restart Easily add and delete nodes
4

The Ultimate Parallel Architecture

Oracle Database 10g RAC

Existing Apps
Financials, MFG, HR and CRM Collaboration Suite In house developed DSS ISV Apps Real Applications in a Real Grid!
ERP CRM DW

Easy Migration Improve Utilization

5

Oracle Database 10g RAC

RAC IPC
Thousands of processes 200K+ associations (not connections) 64 nodes

Oracle IPC Usage

New database functionality will significantly increase IPC utilization
Approach database I/O rates Very large messages (8 MB +)
6

What is RDS?
Vision
A low overhead, low latency, high bandwidth, ultra reliable, supportable, IPC protocol and transport system
Which matches Oracles existing IPC models for RAC communication Optimized for transfers from 200 bytes to 8meg

What is RDS?
Goal and Objective
Support for a reliable datagram IPC in OpenIB
Based on Socket API Minimal code change / testing for Oracle Failover inter HCA and intra HCA ports Runs over IB, Ether, iWARP, etc 2-6 month validation / certification for RAC

What is RDS?
Reliable Datagram IPC
UDP Oracle adds reliable delivery via user mode wire protocol engine
Two sockets per process, thousands of messages on wire Slow sends times (windowing,acks,retrans) Holds together but degenerates under CPU load Well tested !

What is RDS?
Available Options
uDAPL / itAPI not supporting IPOIB high CPU overhead, same unreliable delivery (UDP) SDP connection oriented We want to take our existing well tested UDP module, shutoff most of it to run over an O/S provided RD IPC

10

What is RDS?
RD Reliable Datagram IPC over IB
50% less CPU than IPOIB, UDP Latency of UDP (no user-mode acks) Within 5% of uDAPL thru-put using Oracle Minimal code change reduced our UDP module by 70% - removed windowing, acks, retransmissions, etc. RDS driver ~ = 1k C lines (b-copy) Decoupled from user-mode CPU loading Passes all Oracle regression tests in < 2 wks !!!! Supports fail-over across and within HCAs

11

What is RDS?
RDS IPC over IB
Uses IB reliable connection (RC) Node to Node level connection
User mode sockets share small pool of node to node RCs. Formed either dynamically at send or at system startup

12

Oracle Block Service Rate

00000

00000

00000

rd s ip o i b ge

00000

0000

0 0 0 0 0 0 0 00 00 00 00 00 00 00 00

13

Service Response Time

0 00 . 00 . 00 . 00 . 0 00 . 00 . 00 . 00 . 0 0 0 0 0 0 0 00 00 00 00 00 00 00 00 rd s ip o i b ge

14

CPU Cost Per Block Served

0 000 . 0 000 . 0 000 . 0 000 . rd s 0 000 . 0 000 . 0 000 . 0 000 . 0 0 0 0 0 0 0 00 00 00 00 00 00 00 00 ip o ib ge

15

Open Source RDS for Linux

Open Source RDS
Oracle is building Open Source RDS for Linux
http://oss.oracle.com/projects/rds/

Oracle will support RDS on Linux Oracle RDS will be pulled into OFED Oracle RDS will support GigE, simple NICs and RDMA NICs (IB/iWARP)

16

Open Source RDS for Linux

Status
Oracle is testing Open Source RDS on GigE RDS on IB is in development Beta testing with customers using SilverStorm RDS/IB stack
Very stable Very good performance

RDS is supported by major tier one system vendors

17

Open Source RDS for Linux

OpenFabrics Components and Architecture
Application Level
SA IP Based App Access Sockets Based Access (IBM DB0 ) Various MPIs Block Storage Access Clustered DB Access (Oracle 0 g RAC) 0 Access to File System s Subnet Administrator Management Datagram Subnet Manager Agent Performance Manager Agent IP over InfiniBand Sockets Direct Protocol SCSI RDMA Protocol (Initiator) iSCSI RDMA Protocol (Initiator) Reliable Datagram Service User Direct Access Programming Lib Host Channel Adapter RDMA NIC MAD

User APIs Upper Layer Protocol

Diag Tools Tools

Open SM SM SDP Library

SMA

UDAPL UDAPL
PMA

User Level MAD API

User Level Verbs / API

Use r Sp ce a Ken lSp ce re a

IPoIB SDP

IPoIB

SDP

SRP

iSER

RDS

NFS -RDMA RDMA RPC

Cluster File Sys

SRP

iSER

Connection Manager Abstraction (CMA)

RDS

Mid -Layer

SA Client

MAD

SMA SMA

Connection Manager

Connection Manager R- NIC Driver API

UDAP L HCA

InfiniBand Verbs / API

R -NIC

Provider

Hardware Hardware Specific Driver Specific Driver

Hardware Specific Hardware Specific Driver Driver

Key

Common InfiniBand

Hardware

InfiniBand HCA

ARPR-N NIC IC iW

iWARP

Apps & Access Methods for using OF Stack

Intel Confidential

0 0

18

Beta Customer Experience

Customer Requirements
Improve application performance (throughput, latency, etc.) Maintain data availability Lower TCO through commodity hardware without sacrificing performance or scalability Want to implement Grid and Utility computing

19

Beta Customer Experience

Results
RDS/IB shows significant real world application performance gains for certain workloads: DSS and mixed DSS/OLTP workloads
Throughput and latency

Customers are interested in unified fabric for cost and manageability reasons
Reservation/QoS is a concern

Joint customer Oracle World presentation on RDS (Session S281216, 10/23/2006 11AM)

20