Architecting an Highly Available and Scalable WordPress Site in AWS

Harish Ganesan Co founder & CTO 8KMiles

www.twitter.com/harish11g http://www.linkedin.com/in/harishganesan

Why this presentation ?

WordPress is used by over 14.7% of Alexa Internet's "top 1 million" websites As of August 2011 WordPress powers 22% of all new websites As of December 2011, version 3.0 had been downloaded over 65 million times Provide some architectural insights to build Highly Scalable and Available WordPress sites in AWS Help startups and companies who are new to AWSWordPress with some blueprints

WordPress Architecture Requirements in AWS

Ability to Serve millions of Pages per day Ability to scale out/down dynamically depending upon traffic Ability to perform well even with scaling demands Should be Highly available without Single Points of Failure Automated Operations Backup and Monitoring

Cost effective

Why AWS is better for WordPress Scaling?

AWS provides flexibility to Scale up and Scale out AWS provides Building block services which is inherently robust and fault tolerant AWS provides full range of options from CDN , OS flavors , Storage Pool , Database , Load Balancers , Monitoring etc Pay as you go Save cost by leveraging Auto Scaled WordPress EC2 instances We can start small and grow big in AWS

AWS Building Blocks used in this WordPress Architecture

Some AWS building blocks

Amazon Route 53

Amazon Route53 for managing and answering DNS queries

Amazon Elastic Load Balancer for load balancing HTTP/S requests to WordPress Instances RDS MySQL Database Server for storing the WordPress content

S3

Simple Storage Service (S3) for storing the WordPress application assets , snapshots and log files
Amazon CloudFront CDN for delivering the application static assets from nearest edge locations Amazon CloudWatch for monitoring and sending SNS alerts

Amazon CloudFront

Amazon CloudWatch

Highly Scalable Architecture for WordPress application

Admin
AZ : Amazon Availability Zones

Amazon Route 53
AWS Region 1

Amazon CloudFront

Static Data Elastic Load Balancer

Az-1 Az-2

SSH/SFTP

S3

Amazon CloudWatch Alarms

WordPress Management Server

Amazon SNS Notifications

Auto Scaled WordPress EC2 instances

Az-1 Az-1 Az-2

RDS Master

Az-2
Standby

Read Replica

Backups and Log files

File Storage Pool

Database Layer

Database Layer

Brief about the Architecture

Multi- Tiered Architecture

Load balancing tier Separate WordPress Management and Instance nodes Auto Scaled WordPress EC2 instance nodes MySQL M-S database with additional Read Slaves

S3 + CDN for content delivery

Monitoring , Backups , Sys alerts

Architectural Tiers Explanation

Now let us see the tiers in detail !!

Load Balancing Tier

Admin
AZ : Amazon Availability Zones

Amazon Route 53
AWS Region 1

Amazon CloudFront

Static Data Elastic Load Balancer

Az-1 Az-2

SSH/SFTP

S3

Amazon CloudWatch Alarms

WordPress Management Server

Amazon SNS Notifications

Auto Scaled WordPress EC2 instances

Az-1 Az-1 Az-2

RDS Master

Az-2
Standby

Read Replica

Backups and Log files

File Storage Pool

Database Layer

Database Layer

Load Balancing with Amazon ELB

Amazon Elastic Load Balancing will Load balance the http requests to WordPress EC2 instances ELB will direct the requests to WordPress EC2 across Multiple Availability Zones ( for HA) Health check , LB algorithms and SSL termination will be done by the Load Balancer Amazon Elastic Load Balancing can keep expanding its capacity automatically depending upon the traffic

Load Balancing with Amazon ELB

Amazon ELB works with Route 53 and Auto Scaling seamlessly Amazon ELB is priced @ Pay for use model Amazon ELB is a managed service , so
NO Maintenance headaches

NO SW/HW upgrades
NO Capacity planning NO manual intervention for expansion

How scalable is this architecture ?

Admin
AZ : Amazon Availability Zones

Amazon Route 53
AWS Region 1

Amazon CloudFront

Static Data Elastic Load Balancer

Az-1 Az-2

SSH/SFTP

S3

Amazon CloudWatch Alarms

WordPress Management Server

Amazon SNS Notifications

Auto Scaled WordPress EC2 instances

Az-1 Az-1 Az-2

RDS Master

Az-2
Standby

Read Replica

Backups and Log files

File Storage Pool

Database Layer

Database Layer

Scalability and Elasticity

Scalability and Elasticity is built in most of the layers in this architecture

WordPress EC2 instances can be scaled out and down depending upon the traffic
We can expand the number of WordPress EC2 instances from 1 to 100+ automatically during load peaks We can reduce the number of WordPress EC2 instances from 100+ to 1 automatically during valleys Can serve millions of pages with ease Pay only for the EC2 instance hours used

Scalability and Elasticity

Read Scalability is built in the Database layer with RDS Read Replicas

New RDS Read Replicas can be added / removed with ease

Tips:
During Campaigns alone we can run more RDS Read Replicas
Not more than max 5 RDS Read replicas are recommended Keep Read Replicas and RDS Master same EC2 size for better performance

Scalability and Elasticity

Clustered + Distributed File System of the Storage pool layer can be manually scaled in case needed
Minimum 2 EC2 Large instances should be used for Storage Pool Layer ( for HA and better IO)
Design the Storage Pool Layer with HA (very critical)

AWS building blocks like S3 , CloudFront , CloudWatch , SNS used in this architecture are inherently designed for scalability

How High Availability and Fault tolerance is built in this Architecture ?

High Availability

HA @ WordPress Layer
Multiple WordPress EC2 instances avoid single point of failure WordPress EC2 instances are launched across multiple AZs inside a region for High Availability

HA @ DB Layer
RDS MySQL Master and Standby are launched in 2 different availability zones for High Availability

RDS Read Replicas are created in Multiple AZs

High Availability

HA @ Storage Pool Layer

Two EC2 instances used for Storage Pool Layer

Storage Pool is setup in replicated mode for High Availability

AWS building blocks like S3 , Cloud Front , CloudWatch , ELB , SNS ,EBS used in this architecture are inherently designed for Fault tolerance and HA

How the Database Tier is Architected ?

Admin
AZ : Amazon Availability Zones

Amazon Route 53
AWS Region 1

Amazon CloudFront

Static Data Elastic Load Balancer

Az-1 Az-2

SSH/SFTP

S3

Amazon CloudWatch Alarms

WordPress Management Server

Amazon SNS Notifications

Auto Scaled WordPress EC2 instances

Az-1 Az-1 Az-2

RDS Master

Az-2
Standby

Read Replica

Backups and Log files

File Storage Pool

Database Layer

Database Layer

Database layer

RDS Master and Standby provide High availability RDS Read Replicas provide Read performance

HyperDB plugin is configured to use multiple endpoints like RDS master and RDS read replicas in this architecture DB security groups will allow DB access only to WordPress EC2 instances
Periodic Dumps , Snapshots and Point in time recovery is possible in this architecture

What performance aspects are taken care in this Architecture ?

Performance

APC (or) Xcache plugin can be used for PHP opscode caching W3TotalCache+CloudFront (or) BatCache+Memcached can be used for Page caching

Multiple RDS Read Replicas for Read performance

RDS Master and Read Replicas are separated to get independent write and read performance

Amazon ELB + Auto Scaling improves the overall Site performance when the load increases

Content Delivery Network

Admin
AZ : Amazon Availability Zones

Amazon Route 53
AWS Region 1

Amazon CloudFront

Static Data Elastic Load Balancer

Az-1 Az-2

SSH/SFTP

S3

Amazon CloudWatch Alarms

WordPress Management Server

Amazon SNS Notifications

Auto Scaled WordPress EC2 instances

Az-1 Az-1 Az-2

RDS Master

Az-2
Standby

Read Replica

Backups and Log files

File Storage Pool

Database Layer

Database Layer

Content Delivery

Amazon CloudFront will be the Content Delivery Network (CDN) W3TotalCache plugin will be configured to use Amazon CloudFront for enhanced performance and reduced latency

Static assets , templates , themes , images etc will delivered from the nearest edge locations of the CDN

Deployment

Distributed File Storage Pool is configured between WordPress Management and Content instances WordPress Management and Content instances share the common storage pool for files and plugins Deployment of files and plugins will happen through the WordPress management node Files and plugins will be immediately available for use in the auto scaled WordPress EC2 instances

Monitoring and Alerts

Admin
AZ : Amazon Availability Zones

Amazon Route 53
AWS Region 1

Amazon CloudFront

Static Data Elastic Load Balancer

Az-1 Az-2

SSH/SFTP

S3

Amazon CloudWatch Alarms

WordPress Management Server

Amazon SNS Notifications

Auto Scaled WordPress EC2 instances

Az-1 Az-1 Az-2

RDS Master

Az-2
Standby

Read Replica

Backups and Log files

File Storage Pool

Database Layer

Database Layer

Monitoring

Amazon CloudWatch will monitor the CPU and Network utilization of the entire setup Amazon CloudWatch alarms configured with Amazon SNS provides Email/SMS alerts to System Administrators

Backups

Admin
AZ : Amazon Availability Zones

Amazon Route 53
AWS Region 1

Amazon CloudFront

Static Data Elastic Load Balancer

Az-1 Az-2

SSH/SFTP

S3

Amazon CloudWatch Alarms

WordPress Management Server

Amazon SNS Notifications

Auto Scaled WordPress EC2 instances

Az-1 Az-1 Az-2

RDS Master

Az-2
Standby

Read Replica

Backups and Log files

File Storage Pool

Database Layer

Database Layer

Backup

Custom ops scripts will backup the necessary files from Storage pool to S3 periodically S3 can be configured to remove old backups automatically RDS MySQL is configured to take periodic data dumps and DB snapshots RDS layer can be recovered point in time from the backups

Security

Suitably hardened OS for WordPress Mgmt and WordPress instances Firewall (or) AWS security groups configured between all the layers in the architecture SFTP/FTP access only to the WordPress management node AWS IAM policies to manage user account access Install WordPress Security plugins

Advantages

Highly Available and Scalable architecture Can elastically scale out to serve millions of hits in a day Can grow with load demands in future Usage of inherently fault tolerant AWS building blocks adds Stability High performance using CDN and suitable cache plugins Monitoring , Backup and Recovery is built in Pay for use

Disadvantages

Complex to setup and maintain Will not be ideal for smaller sites that do not need scale Will not be cost efficient for sites that have less traffic

Key Points to Remember

Log files generated in WordPress EC2 instances have to rotated Use Amazon EBS for Storage Pool and WordPress EC2 instances Start with EC2 m1.Large Instances for the storage pool and WordPress instances More Memory Better Performance for DB
Start with RDS Large for Master Keep RDS Master and RDS Read Replicas in same size to improve read performance

Key Points to Remember

RDS MySQL supports only Innodb engine Separate the WordPress Management and WordPress Content instances for scalability Do not Scale out/down rapidly in a hour, it will cost more in AWS

Combine AWS On-Demand and Reserved Instance pricing to get more savings per month

Key Points to Remember

Amazon Availability Zones(AZs) are distinct physical locations with Independent power , cooling ,network and security having Low latency network connectivity between them inside the same region
Leverage them in WordPress , Storage Pool and DB layers for HA as mentioned in the architecture

How do I setup Scalable WordPress architecture on AWS?

Leave it to the experts , we will handle this

Cloud Architecture Consulting

Cloud Application Development Cloud Migration & Implementation Cloud Adoption Strategy

Let's get the job done

Why 8KMiles ?
Amazon Systems Integrator and Solution Developer In-depth understanding of Cloud infrastructure services

8KMiles Media Coverage

Contact Us
All you need is an idea and the cloud will execute it for you. (Structure 2010 event) - Dr Werner Vogels , CTO of Amazon on 8KMiles

For more details on how 8KMiles Cloud Consulting can help your business , contact

cloud@8kmiles.com http://cloud.8kmiles.com http://cloudblog.8kmiles.com http://www.8kmiles.com