Architecture for Grid

Tein Horng Yuan tyuan@beta.wsl.sinica.edu.tw 2003/02/24

1

Why Grid
Large amounts of data in petabyte / Hardware / Software over the Internet Institutions / Individuals around the world need to collaborate together to do the problem solving in a dynamic and flexible way

Virtual Organization
VOs enable disparate groups of organizations and/or individuals to share resources in a controlled fashion, so that members may collaborate to achieve a shared goal. (ref .1)

Grid Architecture
Fabric Layer Connectivity Layer Resource Layer Collective Layer Application Layer

Layered Grid Architecture

Application Collective Resource Connectivity Fabric
5

High-Throughput Computing System

App High Throughput Computing System job management,

API SDK C-point Protocol Checkpoint Repository

ollective Dynamic checkpoint, (App) failover, staging

ollective Generic) Brokering, certificate authorities

API SDK Access Protocol Compute Resource 6

Resource Access to data, access to computers, access to network performance data

Connect Communication, service discovery (DNS), authentication, authorization, delegation Fabric Storage systems, schedulers

Data Grid Architecture

App Discipline-Specific Data Grid Application

ollective Coherency control, replica selection, task management, (App) virtual data catalog, virtual data code catalog,

ollective Replica catalog, replica management, co-allocation, Generic) certificate authorities, metadata catalogs, Access to data, access to computers, access to network performance data,

Resource

Communication, service discovery (DNS), Connect authentication, authorization, delegation Fabric Storage systems, clusters, networks, network caches,
7

EDG middleware architecture Globus hourglass

Current EDG architectural functional blocks: Basic Services ( authentication, authorization, Replica Catalog, secure file transfer,Info Providers) rely on Globus 2.0 (GSI, GRIS/GIIS,GRAM, MDS)

Specific application layer VO common application layer

GRID middleware

ALICE

ATLAS

CMS

LHCb

Other apps

LHC

Other apps

High level GRID middleware Basic Services OS & Net services

8

GLOBUS 2.0

The DataGrid Project

http://eu-datagrid.web.cern.ch/eu-datagrid/ WP1 (Workload Management System) WP2 (Data Management) WP3 (Information and Monitoring Services) WP4 (Fabric Management) WP5 (Mass Storage Management) WP6 (Testbed and Demonstrators) WP7 (Networking) Applications
WP8 (High Energy Physics) WP9 (Earth Observation) WP10 (Biology)

WP11 (Dissemination) WP12 (Project Management)

9

DataGrid
Local Computing

Architecture
Local Database Local Database Metadata Metadata Management Management Object to Object to File File Mapping Mapping

Local Application Local Application

Grid

Grid Application Layer Grid Application Layer

Job Job Management Management Data Data Management Management

Collective Services Collective Services

Information Information & Monitoring & Monitoring Replica Replica Manager Manager Grid Grid Scheduler Scheduler

Underlying Grid Services Underlying Grid Services

Database Database Services Services Computing Computing Element Element Services Services Storage Storage Element Element Services Services Replica Replica Catalog Catalog Authorization Authorization Authentication Authentication & Accounting & Accounting Logging & Logging & BookBookkeeping keeping

Grid

Fabric

Fabric services Fabric services

Resource Resource Management Management Configuration Configuration Management Management Monitoring Monitoring and Fault and Fault Tolerance Tolerance Node Node Installation & Installation & Management Management Fabric Storage Fabric Storage Management Management

10

Client Applications
Web Command line D0 Framework C++ codes Python codes, Java codes

Request Formulator and Planner

Request Manager

Cache Manager

Job Manager

Storage Manager

Connectivity and Resource

SAM-specific user, group, node, station registration

Authentication and Security Fabric

Tape Storage Elements Disk Storage Elements Compute Elements LANs and WANs Code Repostory Resource and Services Catalog Replica Catalog Meta-data Catalog

Collective Services

Dataset Editor

Project Master

Station Master

Station Master
Job Services

File Storage Server

Data Mover

SAM Resource Management

Batch Systems - LSF, FBS, PBS, Condor

Optimiser
Significant Event Logger Naming Service Catalog Manager

Stager
Database Manager

CORBA

UDP

Catalog protocols

File transfer protocols ftp, bbftp, rcp

GridFTP

Mass Storage systems protocols e.g. encp, hpss

GSI

Bbftp cookie

Indicates component that will be replaced

enhanced

or added

using PPDG and Grid tools

Name in quotes is SAM-given software component name

GriPhyN/PPDG Data Grid Architecture

12 From: GriPhyN/PPDG Data Grid Architecture, Toolkit, and Roadmap V2

References
1. 2. Anatomy of the Grid: Enabling Scalable Virtual Organizations http://www-fp.globus.org/research/papers/anatomy.pdf Grid Architecture http://www.globus.org/training/gridarchitecture/GridArchitecture.pdf EDG Tutorial Architecture Overview http://hep-proj-grid-tutorials.web.cern.ch/hep-proj-gridtutorials/presentations/EDG Tutorial - Architecture Overview.ppt GriPhyN/PPDG Data Grid Architecture, Toolkit, and Roadmap V2 http://www.ppdg.net/docs/WhitePapers/DGA 2.09.pdf DO-SAM and the Particle Physics Data Grid http://www.hicb.org/hicb/HICB-011007/Bos-D0-Grid-Ideas.ppt
13

3.

4. 5.

CV
Tein Horng Yuan Academia Sinica Computing Centre PC Farm project leader since 1997 Oct.

14