Professional Documents
Culture Documents
OpenWorld 2015
200 Million QPS on Commodity Hardware
Getting Started with MySQL Cluster 7.4
Frazer Clement
MySQL Cluster Technical Lead
Bernd Ocklin
Director, MySQL Cluster Engineering
October 26th, 2015
Customer Experiences
NEC et al
Conference Session
Conference Session
Benedita Vasconcelos,
Oracle
Hands On Lab
Tuesday 11am.
Moscone South, 262
Tuesday 5.15pm
Moscone South, 250
Thursday, 9.30am
Hotel Nikko - Peninsula
At the corner of Howard and 4th st.; only 2-min walk from Moscone Center
(same place as last year)
Join us!
Copyright 2015, Oracle and/or its affiliates. All rights reserved. |
Telecoms
Other
User Profile
Management
Session Management &
Caching
Service Delivery
Platforms
Mobile Payments
DBMS research
Real-Time
Responsiveness
On-Line, Linear
Scalability
Low TCO,
Open platform
10
SQL
, Row locks, Triggers, Views,
Stored procedures, Blobs, keyless tables, newSQL, MySQL compatible... connectors for most
languages, ORMs etc...
NoSQL Full C++ Api for best control and performance (MySQLD SE built on top), Other Apis :
Java, JPA, Node.js, Memcache....
HA 99.999% uptime systems (five nines),
11
logs
Manageability Online add + drop (index, column) , Online consistent backup, Online
upgrade, Online OS or hardware upgrade, consolidated cluster logs, C management Api for tooling...
Shared nothing, Commodity No need for shared storage, In-memory data uses disk frugally,
TCP over Ethernet / Infiniband etc, No special layer 2 requirements.
Open source.
12
7.2
2012
2013
- Distributed
parallel joins
- Multi-TC
- Active-Active
- Memcached
- MySQL Server
5.5
7.3
2014
- Foreign keys
- Client lib
performance
- node.js
- MySQL
Server 5.6
7.4
2015
- Restart
performance
- Active-Active
- Internal
reporting
- MySQL
Server 5.6
...
13
Performance
optimisations in the
data node kernel
Active-Active
replication
enhancements
Improved
observability and
manageability
14
JPA
REST
Application Nodes
NdbApi protocol
F2
Node Group 2
Node Group 1
F1
Node Group 3
F1
F2
F3
F4
F5
F6
F6
F4
F5
F6
Table 1
F1
F2
F3
F3
F4
F5
Cluster
Mgmt
Data
Nodes
Cluster
Mgmt
15
Redundant
components
P
C A
NdbApi protocol
F2
Node Group 2
Node Group 1
F1
Node Group 3
F1
F2
F3
F4
F5
F6
F6
F4
F5
F6
Table 1
F1
F2
F3
F3
F4
F5
Cluster
Mgmt
Data
Nodes
Cluster
Mgmt
16
Redundant
components
NdbApi protocol
F2
Node Group 2
Node Group 1
F1
Node Group 3
F1
F2
F3
F4
F5
F6
F6
F4
F5
F6
Table 1
F1
F2
F3
F3
F4
F5
Cluster
Mgmt
Data
Nodes
Cluster
Mgmt
17
REST
Application Nodes
NdbApi protocol
Node Group 1
F1
F2
F3
F4
F5
F6
Table 1
Cluster
Mgmt
Node Group
2
Redundancy
for availability
Node Group 3
- API nodes are stateless and consistent, can use n + m sparing with
F2
F3
F1
simple front end
load balancing.
F5and back on data node failures.
F6
F4
- NdbApi automatically
balances, fails over
- Network needs no SPOF too no single failure takes out > 1 cluster
F5
F6
F4
member.
F1
Data
Nodes
F2
Cluster
Mgmt
F3
18
JPA
REST
Application Nodes
NdbApi protocol
F2
F3
F4
F5
F6
Table 1
Node Group 2
Node Group 1
F1
Cluster
Mgmt
Node Group 3
F2
Performance + Capacity
F5
F4
Online
scale out of back end by adding
whole node groups
(Read + Write scaling)
F5
F4
F1
F1
Data
Nodes
F2
F3
F6
Cluster
Mgmt
F6
F3
19
Application Nodes
NdbApi protocol
F2
F3
F4
F5
F6
Table 1
Node Group 2
Node Group 1
F1
Cluster
Mgmt
Node Group 3
F2
Performance + Capacity
F5
F4
Online
scale out of back end by adding
whole node groups
(Read + Write scaling)
F5
F4
F1
F1
Data
Nodes
F2
F3
F6
Cluster
Mgmt
F6
F3
20
Request processing
threads
TC instances
Shared nothing
LDM instances
Shared nothing
TC
Transaction
coordinator
LDM
Local data
manager (Table +
Index partitions)
ndbmtd
Replication thread
Connect threads
Send threads
Receive threads
Watchdog
IO threads
Generally no more than one request processing thread per [HT] core
Copyright 2015, Oracle and/or its affiliates. All rights reserved. |
21
Request processing
threads
TC instances
Shared nothing
LDM instances
Shared nothing
TC
Transaction
coordinator
LDM
Local data
manager (Table +
Index partitions)
ndbmtd
Replication thread
Connect threads
Send threads
Receive threads
Watchdog
IO threads
22
Clients
Clients
Clients
Mysqld
Memcached
Many* threads
Protocol decoding
Business logic / State machines
Database / Persistence layer
API conn
NdbApi calls
libndbclient
NdbApi
API conn
Node.js*
Java
Slapd
...
API conn
'Protocol 6'
Copyright 2015, Oracle and/or its affiliates. All rights reserved. |
23
Clients
Clients
Clients
Mysqld
Memcached
Many* threads
Protocol decoding
Business logic / State machines
Database / Persistence layer
API conn
NdbApi calls
libndbclient
NdbApi
API conn
Node.js*
Java
Slapd
...
API conn
'Protocol 6'
Copyright 2015, Oracle and/or its affiliates. All rights reserved. |
24
Coordination
avoidance
Local
efficiency
Protocol design,
optimisation, packing,
mulitplexing.
Data Distribution
awareness
Locality - Pushed down
filtering and joining
OS call amortisation
Non blocking execution
Cache friendly data
structures
Lock free shared data
structures
Local data structures
Multi granularity pools
Scale Out
Balance
Hash partitioning
Scale Up
See MySQL Connect 2012 session 'Breakthrough performance with MySQL Cluster'
Copyright 2015, Oracle and/or its affiliates. All rights reserved. |
25
SQL joins,
aggregates
Higher volume,
simpler, smaller
footprint
26
Jun 2013
8.5x better
performance
per NdbApi
connection
Jul 2012
1 billion writes
per minute
7.2
7.3
Feb 2015
200 million
NoSQL reads
per second
2.5 million
SQL
statements
per second
50% better
Sysbench read
performance
7.4
27
7.2
NdbApi
Connection
thread
contention
reduction
7.3
Data node
Scan + PK
lookup
optimisations,
Send + Recv
optimisations
...
7.4
28
29
30
32 Data node
machines
running ndbmtd
1 Management node
31
32 Data node
machines
running ndbmtd
384 TC threads
384 LDM threads
1 Management node
32
flexAsynch
The
Infiniband
CloudTM
10 million
conc. reads
72 x 256
threads
72 x 3 API
connections
32 x 12 TC +
LDM threads
> 100 GB
data
33
flexAsynch
The
Infiniband
CloudTM
34
flexAsynch
The
Infiniband
CloudTM
Distribution aware,
minimal hops
Copyright 2015, Oracle and/or its affiliates. All rights reserved. |
35
flexAsynch
The
Infiniband
CloudTM
Distribution aware,
minimal hops
Batching of
requests
36
flexAsynch
The
Infiniband
CloudTM
Distribution aware,
minimal hops
Batching of
requests
Partitioned
client threads
Copyright 2015, Oracle and/or its affiliates. All rights reserved. |
37
Software configuration
Data nodes :
- 12 LDM threads (non-HT)
- 12 TC threads (HT)
- 2 Send threads (non-HT)
- 8 Receive threads (HT)
- MaxSendDelay config
API nodes :
- 3 NdbApi connections per client
machine
- 256 flexAsynch threads per client
machine
Scripts : https://dev.mysql.com/downloads/benchmarks.html
38
250
200
160
140
Million reads/s
Million reads/s
120
150
100
100
80
60
40
50
20
0
0
10
15
20
25
30
35
0
0
20
40
60
80
100
120
140
160
180
39
QuickStart whitepaper
http://downloads.mysql.com/tutorials/cluster/mysql_wp_cluster_quickstart.pdf
40
My laptop
F1
F4
F4
F1
41
Classroom
Training
Cloud
Learning
Technology
Subscription
Applications
Industries
Training On
Demand
education.oracle.com
Copyright 2015, Oracle and/or its affiliates. All rights reserved. |
42
Customer Experiences
43
44