CON3846

MySQL Server and MySQL Cluster at India

s Financial Inclusion Gateway Service
27 th Oct 2015
NEC Corporation

Agenda
1. Project overview
2. Our Methodology
3. System Architecture
Of Financial Inclusion Gateway System
4. Approach of System Design
5. Conclusion & QA

1. Project Overview

Background of Financial Inclusion

Financial inclusion is a key enabler to reducing
poverty and boosting prosperity.

NEC Corporation 2015

NEC Group Internal Use Only

Issue of Financial inclusion

World Average

High Income area

94%

51%
Middle East

14%
South Asia

Sub-Saharan Africa
Latin America and
Caribbean

62%

Central Asia & Europe

46%

East Asia & Pacific

69%

34%

51%

Source : Worldbank, Global Financial Inclusion Index 2014

5

NEC Corporation 2015

NEC Group Internal Use Only

<VIDEO>
Our Solution Overview

NEC Corporation 2015

NEC Group Internal Use Only

Sustainable
Earth

Safer Cities & Public

Services

Quality of Life

Communication

Work Style
7

NEC Corporation 2015

Lifeline
Infrastructure

Industry Eco-System
NEC Group Internal Use Only

Value Creation through ICT

Value Creation
Through ICT
Secured Cyber Field

Solutions
Data

NEC Corporation 2015

Value
Value
Creation
Creation

Security
Security

NEC Group Internal Use Only

Solutions
Data

NECs involvement in the MySQL community

NEC Members in discussion with CORE

Members of MySQL development

NEC Corporation 2015

NEC Group Internal Use Only

NEC Members in front of the

Oracle Office @Stockholm

Financial Inclusion Gateway

Financial Inclusion Gateway

System Configurator
- Product Config,.
3rd Party, Static Data

Txn Engine- A/C opening,

Payment Txn, Collections

Reconciliation &
Clearing Engine

Data Layer (Transactional & Configuration)

Common Features
Platform Services
Platform Monitoring

Mobile terminal
10

NEC Corporation 2015

Core
Core
Banking
Banking
System
System

Gateway Engine

Biometric
Auth.

Txn / Business Validation

Automatic / Manual
Transaction Reversal

CBS
Connectivity

Dashboard ServicesAdmin / BC.Txn

History / Reports

ISO8583
Message
Handler

Internet
Connectivity

Business
Correspondent
Management

ON-US /
Customer Mgt.
CBS Message
Handler

Secure
connection

Txn Acquisition Layer / Channel

Handler

KIOSK

Audit / Transaction
Trails

Secure Login / Session

Role

Platform & Transaction

Security

Business / Technical
Exceptions

NEC Group Internal Use Only

National
National
ID
ID
E-KYC /
Authentication

Conditions in India
Condition
Condition

Our
Our design
design policy
policy
Asynchronous Process

Poor Network Environment

Unstable network connection

High fault tolerance

Shared nothing Architecture

Huge Population
The number of requests which
can't be predicted

Low Cost per Transaction

Low-price rate charge

11

NEC Corporation 2015

High expandability scalability

Open Source Software

Reduction in license cost

NEC Group Internal Use Only

2. Our Methodology

NEC and Open Mission Critical System (OMCS)

NEC has been involved in the development of OMCS for the following sectors:
Mega-banks
Airlines
Government Institutions

6 keywords in OMCS platform development

Mission Critical (MC) Property
MC Facility
System Metrics
Multi-Unit Architecture
Unit method of construction
System Model
13

NEC Corporation 2015

NEC Group Internal Use Only

MC Property
There
There are
are 6
6 elements
elements in
in MC
MC property.
property.
High-Availability
High-Availability

High-Performance
High-Performance

High-Expandability
High-Expandability

High-Confidentiality
High-Confidentiality

High-Connectivity
High-Connectivity

High-Operability
High-Operability

MC Property must be considered by all layers, i.e., from

Hardware to Application layer.
HA of Application level
HA of Middleware level
HA of OS level
HA of Hardware level

High-Availability triangle
14

NEC Corporation 2015

NEC Group Internal Use Only

The Importance of System Architecture

1

based on
MC Property

System
Architecture

2
Implementation
model #1

Implementation
model #2

sizing factor
(based on user
requirement)

Architecture Implementation model

= 1:n

It's very important to understand the

inner structure of the candidate
Implementation product by Whiteboxing .
model #3
y=f(x)
y : System
x : Implementation model
f : Sizing factor

3
system

15

NEC Corporation 2015

NEC Group Internal Use Only

3. System Architecture
of Financial Inclusion Gateway System

System Architecture

Application

Internet

Client

API
Layer

Web/
Reverse
Proxy

Abstract
Layer

Logic
Layer
Coordination
Logic

Data
Access
Object

Gov.
Institutions

National ID
Driver

Abstract
Interface

RESTful
API
Data
Access
Logic

BANK
Driver
ClusterJ

JDBC
Driver

Memcached
listener/
sender

Memcached
Driver

(SQL)
(SQL)
Transasction
Transasction
log
log access
access

17

NEC Corporation 2015

Ext.
Services

DB

Internet

Web

NEC Group Internal Use Only

DB
Layer

(noSQL)
(noSQL)
Session
Session
data
dataaccess
access

Banks

3 rd party
services

System Component Diagram

Bank
coordination
service

Data center
Node

Internet

Client

Node

Jetty

Jetty
Load
Balancer

memcached
NGINX

ClusterJ

Node

Data node
ndbmtd
Node
Mgmt node
ndb_mgmd

SQL node
mysqld
MySQL
Cluster

MySQL cluster offers data

access by PKEY.
memcached API or SQL
NEC Corporation 2015

BANK

Async
Node

18

Internet

NEC Group Internal Use Only

Replication
Node

National
ID

MySQL
InnoDB
Complicated SQL
processing uses MySQL
(InnoDB).

System Processing Policy

Back-End

Front-End
Authentication
Authentication request
Authentication

AP

Authentication result

Client

NGINX

Processing request

Banking

Jetty
AP

Processing result
Processing request

Result
confirmation

AP

Generate
a session

MySQL
Cluster

Processing result

Processing request

(Asynchronous)
Reference
the processing result

NEC Corporation 2015

result

Synchronous

NEC Group Internal Use Only

Processing
request

Bank
Banks
coordination
service
Processing

Processing result

Synchronous

19

National
ID

4. Approach of System Design

Use of whiteboxing for the selection of products

The factors to consider before selecting an open source database

Huge transaction amount

Low Latency
Unlimited scalability

The combination of MySQL Cluster & MySQL server was judged to be the
most suitable product.
21

NEC Corporation 2015

NEC Group Internal Use Only

Process of Product estimation by whiteboxing

Understanding of product composition
Understanding of process structure
Visualization with Mevalet
Evaluation of MC property with Mevalet
Executive summary,
Notices and Known Limitations
NOTE: Mevalet is NECs internal performance analysis tool
22

NEC Corporation 2015

NEC Group Internal Use Only

Tracing Target events with Mevalet

Visualization of processes and threads
Wakeup
process

Process Save
(Sleep)

Process#1

CPU
Disk

Using CPU
Waiting for Disk

Disk access
request

Process#2

Process
resume

Waiting for CPU

Using Disk

Disk access
start

System
Call

Process Save
(Preempt)

Using CPU User mode

Disk access
completion

CPU

Inter Process
Communication

Disk
Interrupt
begin

Interrupt
end

Interrupt CPU

23

Return
to User

NEC Corporation 2015

t
NEC Group Internal Use Only

Whiteboxing Example MySQL Cluster 1/2

Mapping Traceview of Mevalet to signal
insert

insert

commit

commit

insert

insert

commit

commit

complete phase
COMPLETE

(Primary NDB Secondary NDB)

COMPLETE
(Secondary NDB Primary NDB)

prepare phase
TCKEYCONF

commit phase
TC_COMMITREQ
insert
(API Primary NDB)

(Primary NDB API)

commit

insert

COMMIT

LQHKEYREQ
(Primary NDB Secondary NDB)

COMMIT

TCKEYREQ

(Secondary NDB Primary NDB)

(API Primary NDB)

TC_COMMITCONF

TCSEIZEREQ

(Primary NDB API)

NEC Corporation 2015

LQHKEYCONF
(Secondary NDB Primary NDB)

(Primary NDB Secondary NDB)

24

commit

(API Primary NDB)

NEC Group Internal Use Only

Whiteboxing Example MySQL Cluster 2/2

Sequence chart: Case of INSERT Operation (Prepare Phase)
mysqld

NDB PRIMARY

API

DBTC
TCSEIZEREQ
TCSEIZECONF
TCKEYREQ

DBTC

Add index to ordered index

DBLQH

ObtainTransaction Record
(Executed at the instance when TC
(Primary Replica) is changed
Inquire the node that stores the
target data

DIGETNODESREQ
LQHKEYREQ
ACCKEYREQ

Create/Obtain hash key of target

data
Add target data to table

NDB SECONDARY

DBLQH

TUPKEYREQ

DBDIH

DBDIH

DBACC

DBACC

DBTUP

DBTUP

TUX_MAINT_REQ
TUPKEYCONF

TUX_MAINT_REQ

Index is
added
LQHKEYREQ

DBTUX

DBTUX
ACCKEYREQ
TUPKEYREQ

LQHKEYCONF
TCKEYCONF

25

NEC Corporation 2015

NEC Group Internal Use Only

TUPKEYCONF

Results of Analysis
PROS

CONS

Performance
- The latency for INSERT operations is much
lower compared to relational databases

Confidentiality
- Data transmission between SQL node and data
nodes is not authenticated and encrypted

Scalability
- Shared Nothing Architecture

Inappropriate for SQL transactions

- Degradation in Performance when executing
complicated SQL execution (SELECT, JOIN)

High-availability
- Redundant MySQL nodes and data nodes

Using Mevalet, we could procure information about the behavior of MySQL Cluster on a real
machine.
1. Timing of disk access
2. Occurrence of CPU WAIT
3. The type of process sequence (serial or parallel)

26

NEC Corporation 2015

NEC Group Internal Use Only

5. Conclusion & QA

Conclusion
NEC carried out market analysis regarding the requirements of the current
financial market of India.

NEC conducted a detailed research on the internal architecture and

behavior of MySQL cluster and concluded that MySQL cluster is the optimal
choice to meet the current market requirements.

By participating actively in the improvement of MySQL cluster, NEC is

aiming to develop more social solutions that utilize MySQL cluster.

28

NEC Corporation 2015

NEC Group Internal Use Only

Reference Slides

System Architecture (High-level)

The system consists of the following 3 layers
Web Layer: It contains the web server and the reverse proxy
AP Layer To receive the request and to store the transaction data, backup data, etc.
into the DB. It also contains the interface to connect the FI- Gateway to External Service
providers.
DB Layer: To store the session info, transaction log. Since the DB is shared-nothing and fullscalable, the response time will remain constant regardless of the new scale

FI -Gateway

Client

PC
terminal
Mobile
device

30

External
Service
Provider
Gov
Institutions

Web
Layer

NEC Corporation 2015

Application
Layer

NEC Group Internal Use Only

DB
Layer

banks
3rd Party
Providers

Memcached with MySQL Cluster

Accessing the MySQL Cluster from memcached
Jetty

Application Java

M emcached Protocol listener/sender

M emcached driver for NDB
NDB API C++

MySQL Cluster
NDB

31

NEC Corporation 2015

NDB

NEC Group Internal Use Only

NDB

Replication of MySQL Cluster

MySQL
MySQL cluster
cluster offers
offers data
data access
access by
by PKEY
PKEY in
in memcached
memcached API.
API. Data
Data of
of MySQL
MySQL Cluster
Cluster is
is copied
copied in
in
MySQL
MySQL (InnoDB)
(InnoDB) by
by the
the replication
replication function.
function.
Complicated
Complicated SQL
SQL processing
processing uses
uses MySQL
MySQL (InnoDB).
(InnoDB).
EC2

EC2

EC2

EC2

Jetty
memcached

EC2

ndb_mgmd
Mgr node#1

MySQL Cluster

ndb_mgmd
Mgr node#2

Replication
Master
Replication
Slave

Jetty
memcached

EC2

ndbmtd

Data node#1
EC2

mysqld

SQL node#1
EC2

mysqld

MySQL srv#1
InnoDB

ndbmtd

Data node#2
EC2

mysqld

SQL node#2
EC2

mysqld

MySQL srv#2
InnoDB

MySQL Server

32

NEC Corporation 2015

NEC Group Internal Use Only

Potential Causes of Various Problems

In
In most
most cases,
cases, problems
problems arise
arise because
because aa basic
basic measure
measure is
is not
not
followed
followed properly.
properly.
Ex. Performance Problems
reee ooff
ooff ddeeggre
ke
ta
is
ke
m
ta
g
is
in
m
tt
e
g
S
Settin
tionn
licaatio
nn aapppplic
a
f
o
g
a
n
f
o
xi
g
le
n
p
xi
lti
u
m
multiple
s..
H
pr
Huuggee
cesss
prooce
ttrraannssac
acttiioonn
vvoolluum
mee
m
a
tt pprrooggrram
n
ie
n
ic
ie
ff
ic
e
in
ineff
ic
lo
loggic

AA ggr
re
ooff llo eaatt ddeeaal
l
ogg oou
uttppuut
t..

ivee
E
Exxcceessssiv
on..
nnoorm
rmaalilizzaatition D
Deessig
ignn m
mis
istak
ooff aa ddaatab takee
vvaar
tabaassee..
r
i
o
i
o
bboot uuss
tttlleen
neec
ckkss

How can we achieve MC Property?

How do we avoid various problems?

The answer is System Architecture.

33

NEC Corporation 2015

NEC Group Internal Use Only