Many users are challenged with how to setup multi-master, multi-site MySQL clusters. How to do you go from a single database server to a scalable cluster? Or move from a brittle MySQL replication system to a transparent, manageable database cluster? In this joint webinar with Continuent, Percona examines how Continuent Tungsten addresses these problems. First, we discuss the read-write splitting functionality of the Tungsten Connector. Next, we demonstrate the multi-data center capabilities of Continuent Tungsten. We set up a total of 9 cluster nodes in 3 different data centers and demonstrate various failure scenarios, while traffic is running: - One of the nodes goes down - Two of the nodes go down - One of the data centers goes down - Two of the data centers go down At the end of this webinar you will know how to quickly configure and provision highly optimized Continuent Tungsten deployments in the cloud or on-premises. Presenters: - Peter Boros, Consultant, Percona - Robert Hodges, CEO, Continuent

1. Multi-Data Center MySQL with
Continuent Tungsten
Peter Boros, Consultant, Percona
Robert Hodges, CEO, Continuent
Percona Technical Webinars
March 2, 2013

2. About This Presentation
• Why is multi-data center MySQL
important?
• How does Continuent Tungsten fill the
gaps?
• Demonstration
• Practical steps to multi-data center
MySQL
www.continuent.com
2
www.percona.com

3. Why is multi-data center
MySQL important?
Robert Hodges

4. Key reasons for multi-data
center MySQL
• High availability – Moving data away from
problems
• Performance – Moving data closer to
users
• Legal – Keeping data within some
locations but not others
www.continuent.com
4
www.percona.com

5. HA Example – Amazon Failure
EBS fails in a single
availability zone
AZ
AZ
AZ
AZ
USAZ
AZ
West-1
AZ
AZ
AZ
AZ
AZ
AZ
AZ
US-East-1
AZ
AZ
EUAZ
AZ
West-1
Availability Zones
Region
www.continuent.com
AZ
AZ
5
www.percona.com

6. HA Example – Amazon Failure
Leading to failures within the region
due to coupled Amazon services
AZ
AZ
AZ
AZ
USAZ
AZ
West-1
AZ
AZ
AZ
AZ
AZ
AZ
AZ
US-East-1
AZ
AZ
EUAZ
AZ
West-1
Availability Zones
Region
www.continuent.com
AZ
AZ
6
www.percona.com

7. HA Example – Amazon Failure
Leading to failure of tightly coupled
applications in other regions
AZ
AZ
AZ
AZ
USAZ
AZ
West-1
AZ
AZ
AZ
AZ
AZ
AZ
AZ
US-East-1
AZ
AZ
EUAZ
AZ
West-1
Availability Zones
Region
www.continuent.com
AZ
AZ
7
www.percona.com

8. Requirements for successful
multi-DC MySQL
• Local HA within geographic regions
• Failover
• Planned maintenance and upgrade
• Advanced multi-site replication
• Flexible topologies
• Tolerant of faults and WAN latency
• Transaction filtering
• Minimal changes to apps / migration
www.continuent.com
8
www.percona.com

9. How does Continuent
Tungsten fill the gaps?
Robert Hodges

10. Tungsten picks up where
MySQL leaves off
• Handle local DBMS server failures
• Perform maintenance while apps are
running
• Load balance SQL across DBMS
• Create disaster recovery sites
• Enable multi-master replication
• Replicate transactions in real-time to
other DBMS types
www.continuent.com
10
www.percona.com

11. Tungsten Cluster Architecture
• Off-the-shelf
MySQL build
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
• Async replicators
• Peer-to-peer
management
www.continuent.com
db1
db3
Manager
Manager
Manager
Replicator
Replicator
Replicator
Slave
• Network-based
connectors
db2
Master
Slave
11
www.percona.com

12. How connectivity works
• Transparent: Implements
MySQL wire protocol
Application Stack
TCP/IP
• Runs anywhere on network
Tungsten Connector
Master
• Routes transactions to master
by default
• Optional read/write splitting
db2
db1
Manager
Manager
Slave
www.continuent.com
• Reads load-balanced across all
slaves
12
www.percona.com

13. Graceful failover
• Triggered by
command
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
• Zero-data loss
• Transparent to
applications
www.continuent.com
db1
db3
Manager
Manager
Manager
Replicator
Replicator
Replicator
Slave
• Automatic
reconfiguration
db2
Master
Slave
13
www.percona.com

14. Hard failover – MySQL crash
• Triggered by
automatic rules
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
• Within seconds
for MySQL crash
• Up to 1 minute
after host crash
• Admin required
for recovery
www.continuent.com
Shunned
db2
db1
db3
Manager
Manager
Manager
Replicator
Replicator
Slave
14
Master
Slave
www.percona.com

15. Hard failover – quorum loss
• Managers
maintain quorum
• Connectivity
stops when lost
• Works across
Amazon AZs
• …But not regions
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
X
X
Shunned Shunned Shunned
db2
db1
db3
Manager
Manager
Failed
Manager
Failed
Replicator
Replicator
Slave
www.continuent.com
X
X
15
Master
Slave
www.percona.com

16. How replication works
•
•
Moves transactions
between points
east
Replicator
Can replicate between
clusters
•
Replicator
Can run multiple services
•
DBMS
Global transaction IDs
•
•
Very flexible topologies
west
Replicator
east
Manager
west
Transaction filtering
www.continuent.com
Replicator
Replicator
16
www.percona.com

17. Cross-site multi-master with
independent clusters
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
Normal
operation
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
west
east
Replicator
east
west
Replicator
Replicator
east
west
Replicator
www.continuent.com
17
www.percona.com

18. Multi-master replicator failure
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
Automatic
rerouting after
replicator failure
west
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
east
Replicator
east
west
Replicator
Replicator
east
west
Replicator
www.continuent.com
18
www.percona.com

19. Multi-master node failure
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
Automatic
failover following
node failure
west
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
east
Replicator
east
west
Replicator
Replicator
east
west
Replicator
www.continuent.com
19
www.percona.com

20. Full site failure
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
West site down for
multiple days
west
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
east
Replicator
east
west
Replicator
Replicator
east
west
Replicator
www.continuent.com
20
www.percona.com

21. Cross-site primary/DR with
composite data service
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
Normal
operation
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
west
east
Replicator
Replicator
Replicator
Replicator
us
www.continuent.com
21
www.percona.com

22. Composite service node
failure
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
Failover to
slave node
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
west
east
Replicator
Replicator
Replicator
Replicator
us
www.continuent.com
22
www.percona.com

23. Composite service site failure
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
Manual failover to
slave site
Application Stack
Application Stack
Tungsten Connector
Tungsten Connector
west
east
Replicator
Replicator
Replicator
Replicator
us
www.continuent.com
23
www.percona.com

24. Demonstration
Peter Boros

25. Demo Architecture
EU-West-1
World
US-East-1
US-West-1
EU-West-1
US-West-1
Composite Data Service
Multi-Master Topology
www.continuent.com
25
www.percona.com

26. Practical steps to multi-data
center MySQL
Robert Hodges

27. Setting Up Multi-DC
Clustering
• Tungsten tpm – Sets up local clusters and
replication links between them
1. Put tungsten.ini file on host
2. Download software (rpm or tarball)
3. Run ‘tpm install’
www.continuent.com
27
www.percona.com

28. Ini File for Composite Service
[defaults]!
home-directory=/opt/continuent!
user=tungsten!
replication-user=tungsten!
replication-password=password!
application-user=sbtest!
application-password=sbtest!
mysql-connectorj-path=/home/tungsten/mysql-connector-java-5.1.26bin.jar!
start-and-report=true!
[eu]!
master=tw-eu1!
slaves=tw-eu2,tw-eu3!
connectors=tw-eu1,tw-eu2,tw-eu3!
[west]!
master=tw-usw1!
relay-enabled=true!
relay-source=eu!
slaves=tw-usw2,tw-usw3!
connectors=tw-usw1,tw-usw2,tw-usw3!
[world]!
composite-datasources=eu,west!
www.continuent.com
28
www.percona.com

29. Composite Data Services –
Pros and Cons
• Key benefits:
• No conflicts
• No app changes
• Failover is transparent
• Works with statement replication
• Things to consider and test:
• High latency writes to primary master
• Managing unplanned site failover
• Potential coupling between sites
www.continuent.com
29
www.percona.com

30. Multi-Master Data Services –
Pros and Cons
• Key benefits
• Sites are completely uncoupled
• No failover!
• Minimal write latency
• Things to consider and test carefully:
• Potential for conflicts
• App changes often required
• Row replication strongly recommended
www.continuent.com
30
www.percona.com

31. Workloads that handle multimaster very well
• Append-only data
• Tables that use only auto-increment keys
• Sharded apps where writes to each shard
are in a particular region (system of
record)
www.continuent.com
31
www.percona.com

32. Where to get started
• Cluster and replication documentation
• https://docs.continuent.com
• Software
• https://code.google.com/p/tungstenreplicator/
• http://www.continuent.com/downloads/
software
• Percona and Continuent Websites
www.continuent.com
32
www.percona.com

33. Thank You!
Peter Boros
peter.boros@percona.com
Robert Hodges
robert.hodges@continuent.com
www.continuent.com
33
www.percona.com