More Related Content Similar to Replication features, technologies and 3rd party Extinction Similar to Replication features, technologies and 3rd party Extinction (20) 1. Replication features, technologies
and 3rd party Extinction.
Percona Live London MySQL Conference, 2012
Andrew Moore Ben Mildren
MySQL DBA MySQL Team Technical Lead
moore@pythian.com mildren@pythian.com
2. Why Pythian?
• Recognized Leader:
• Global industry leader in data infrastructure managed services and
consulting with expertise in Oracle, Oracle Applications, Microsoft SQL
Server, MySQL, big data and systems administration
• Work with over 200 multinational companies such as Forbes.com, Fox
Sports, Nordion and Western Union to help manage their complex IT
deployments
• Expertise:
• One of the world’s largest concentrations of dedicated, full-time DBA
expertise. Employ 9 Oracle ACEs/ACE Directors
• Hold 7 Specializations under Oracle Platinum Partner program,
including Oracle Exadata, Oracle GoldenGate & Oracle RAC
• Global Reach & Scalability:
• 24/7/365 global remote support for DBA and consulting, systems
administration, special projects or emergency response
© 2012 – Pythian
3. What do we cover in this
presentation?
1) MySQL Replication Overview
2) MySQL Replication Topologies
3) MySQL Replication Ecosystem
4) MySQL Replication Tools
5) MySQL Replication in 5.6
6) MySQL Replication Tools in 5.6
© 2012 – Pythian
5. What is replication?
• MySQL Replication is a means of propagating database
changes from one server to another.
• Master / Slave architecture
• MySQL Replication is easy to setup and can be used as the
basis of a number of different functions:
Reporting
Scale out Testing
Failover
Backups
© 2012 – Pythian
13. Binary Log
• The binary log was introduced in version 3.23.14, and records
events to capture changes made to the database.
• DDL, DCL, and DML – includes transaction grouping.
• Not equivalent to a redo log.
• Not just used for Replication, but also for Point in Time
Recovery. It's also a good source of information to use for
auditing and troubleshooting.
• Can be examined using the mysqlbinlog tool or SHOW
BINLOG EVENTS; MySQL statement.
© 2012 – Pythian
15. Binary Log
Statement based logging
• Captures just the SQL statements, means logging is lightweight
and binlog storage requirements are minimized.
• Non-deterministic statements can mean it's prone to data drift.
Row based logging
• Introduced in 5.1
• More verbose, as a result can mean larger binary logs, but also
less data drift.
Mixed format logging
• By default uses statement based logging, but switches to row
based logging for unsafe statements.
© 2012 – Pythian
16. Binary Log
Binary log – Format_desc
[ec2 mysql]$ sudo mysqlbinlog mysqld-bin.000001
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#121112 3:05:24 server id 6 end_log_pos 106 Start: binlog v 4, server v
5.1.61-log created 121112 3:05:24
BINLOG '
xK2gUA8GAAAAZgAAAGoAAAAAAAQANS4xLjYxLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAEzgNAAgAEgAEBAQEEgAAUwAEGggAAAAICAgC
'/*!*/;
• Binlog v4 introduced MySQL 4.1
(v1 – 3.23, v3 – 4.02 - 4.1, v2 – redundant)
• Server version
• end_log_pos: 98 (5.0), 106 (5.1), 107 (5.5), 120 (5.6)
© 2012 – Pythian
17. Binary Log
Binary log - Statement based logging
[ec2 mysql]$ sudo mysqlbinlog mysqld-bin.000001
...
# at 106
...
# at 127924
#121112 3:50:03 server id 6 end_log_pos 128160 Query
thread_id=18112805 exec_time=0 error_code=0
SET TIMESTAMP=1352710203/*!*/;
UPDATE t1 SET c2 = 'ben' WHERE c1 = '1'
/*!*/;
...
© 2012 – Pythian
18. Binary Log
Binary log - Row based logging
[ec2 mysql]$ sudo mysqlbinlog -vvv mysqld-bin.000001
# at 2609282
#121129 0:24:19 server id 6 end_log_pos 2609282 Table_map: `db1`.`t1`
mapped to number 9057
#121129 0:24:19 server id 6 end_log_pos 2609389 Write_rows: table id
9057 flags: STMT_END_F
BINLOG '
g/GUBMGAAAAQAAEjAAAAAAEABWAAPYTHIANROCKSAAAMVyX3Nlc3Npb24ABwMHB/4DDw8G
/kD/ADIAdA==
g/GUBcGAAEAB//kn2U5AIPxtlBAMmM0MTYyYWE2MmQxMTAMAMAMAI4OTdhMxMWMxNHciBwA=
'/*!*/;
### INSERT INTO db1.t1
### SET
### @1=2 /* INT meta=0 nullable=0 is_null=0 */
### @2='andy' /* VARCHAR(10) meta=0 nullable=0 is_null=0 */
### @3=NULL /* TIMESTAMP meta=0 nullable=1 is_null=1 */
### @4=1 /* INT meta=0 nullable=1 is_null=0 */
© 2012 – Pythian
21. Relay Log
• The relay log is a binary log
• end_log_pos relates to master binlog not the event size
[ec2 mysql]$ sudo mysqlbinlog -vvv –base64-output=decode-rows relay-bin.000001
# at 2609427
#121129 0:24:19 server id 6 end_log_pos 2609282 Table_map: `db1`.`t1`
mapped to number 9057
#121129 0:24:19 server id 6 end_log_pos 2609389 Write_rows: table id
9057 flags: STMT_END_F
### INSERT INTO db1.t1
### SET
### @1=2 /* INT meta=0 nullable=0 is_null=0 */
### @2='andy' /* VARCHAR(10) meta=0 nullable=0 is_null=0 */
### @3=NULL /* TIMESTAMP meta=0 nullable=1 is_null=1 */
### @4=1 /* INT meta=0 nullable=1 is_null=0 */
© 2012 – Pythian
23. Replication Filters
• binlog-do-db and binlog-ignore-db - Filters on the master
• Replicate-do-db, replicate-ignore-db, replicate-do-table, and
replicate-ignore-table - Filters on the slave.
• Replicate-wild-do-table and replicate-wild-ignore-table - Filter
on the slave.
• Can have unforeseen consequences, Row based logging tends
to have the best results. Additionally has implication on point in
time recovery.
• Column based filtering – Slave has more/fewer columns
• Row based filtering – Triggers
© 2012 – Pythian
24. Replication Administration
SHOW SLAVE STATUSG
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: repl_host
...
Master_Log_File: mysql-bin.000118 (IO_thread)
Read_Master_Log_Pos: 137492455 (IO_thread)
Relay_Log_File: mysqld-relay-bin.000011 (Relay Log)
Relay_Log_Pos: 131560187 (Relay Log Pos)
Relay_Master_Log_File: mysql-bin.000118 (SQL_thread)
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
...
Exec_Master_Log_Pos: 137492455 (SQL_thread)
...
Seconds_Behind_Master: 0
...
© 2012 – Pythian
26. Communication
Asynchronous replication
• Built in, default, replication method, doesn't require any
additional configuration.
• Lightweight, doesn't wait for any remote confirmations.
• Highest performance, lowest durability.
Semi-Syncronous replication
• Introduced in 5.5, implemented using plug-ins on master and
slave.
• Waits for confirmation that an event from the binary log on the
master has been written to the relay log of at least 1 slave.
• Reverts to asynchronous replication if the confirmation times
out.
© 2012 – Pythian
37. Replication Ecosystem
Overview
Products
Tungsten Replicator
Galera
Master High Availability
Tools
Data Consistency
Prefetching
Maintainence
Monitoring
© 2012 – Pythian
40. Replication Ecosystem
Tungsten Replicator
● External to MySQL Replication (but depends on binlog)
● Cross version (5.1, 5.5), cross fork (Percona, Oracle...)
● Global Transaction IDs
● Parallel Replication
● Heterogeneous Environments (MySQL → Oracle → Postgres, ...)
● Complex topologies, Multiple masters with single slave
● Time-Delay replication
● Auto Provisioning tools
● Enterprise functionality and support available
https://code.google.com/p/tungsten-replicator/
© 2012 – Pythian
43. Replication Ecosystem
Tungsten Replicator
Nice to know
● Relies on the binglog
● Another product to learn, configure, maintain
● New considerations for monitoring (Nagios plugins etc)
Further Reading:
http://code.google.com/p/tungsten-replicator/wiki/TungstenReplicatorCookbook
https://s3.amazonaws.com/releases.continuent.com/doc/replicator-
2.0.4/html/Tungsten-Replicator-Guide/content/index.html
http://datacharmer.blogspot.co.uk
https://launchpad.net/galera/
© 2012 – Pythian
45. Replication Ecosystem
Galera
● Synchronous replication (no slave lag)
● No SPOF
● Simplified high availability using JDBC/PHP connectors
● Multi threaded slaves (row level)
● Consistency guaranteed
● Scalable reads & writes
● True Master-Master Active-Active (read/write on any node)
● XtraDB Cluster, MariaDB support
● Auto Provisioning tools
● Commercial support available
https://launchpad.net/galera/
© 2012 – Pythian
47. Replication Ecosystem
Galera
Nice to know
● InnoDB only
● Synchronicity adds overhead
● As fast as the slowest node
Further Reading:
http://codership.com/content/using-galera-cluster
http://www.severalnines.com/clustercontrol-mysql-galera-tutorial
http://openlife.cc/search/node/galera
https://launchpad.net/galera/
© 2012 – Pythian
49. Replication Ecosystem
MHA
● Not a replication product
● Simplifies Failover
● Takes care of re-aligning slaves
● Elects most up-to-date slave to promote as new Master
● Automated monitoring and failover of Master
● Interactive failover
● Failover without the monitoring or the interactivity
● Online switching masters (think maintenance)
● Commercial support available from SkySQL
http://code.google.com/p/mysql-master-ha/
© 2012 – Pythian
51. Replication Ecosystem
MHA
Nice to know
● No automated failback (not a bad thing!)
● Manager node now a SPOF unless clustered (heartbeat?)
● Author now working at Facebook so ? Around future
development of the tool.
Further Reading:
http://code.google.com/p/tungsten-replicator/wiki/TungstenReplicatorCookbook
https://s3.amazonaws.com/releases.continuent.com/doc/replicator-
2.0.4/html/Tungsten-Replicator-Guide/content/index.html
http://datacharmer.blogspot.co.uk
http://code.google.com/p/mysql-master-ha/
© 2012 – Pythian
53. Eco-Tools
Replication has got it's problems...(currently)
Primary
Data Consistency
Performance
Secondary
Monitoring
Features
© 2012 – Pythian
54. Eco-Tools
Data Consistency
Human Intervention
Performance ● set global sql_slave_skip_counter=1;
Monitoring ● set sql_log_bin=0;
Features
● DML statements executed directly on the
slave
● Writing to both sides of a master-master
topology
Technical Difficulties
● sync_binlog=0 + crash
© 2012 – Pythian
56. Eco-Tools
Data Consistency
Performance INSERT INTO t1 … VALUES (1, ...)
[ERROR] Duplicate entry '1' for key "PRIMARY"
Monitoring
DELETE FROM t1 WHERE date = '2012-12-04 13:45:00'
Features
[ERROR] Slave SQL: Could not execute
Delete_rows
!!!YOUR DATA IS* INCONSISTENT!!!
*probably
© 2012 – Pythian
59. Eco-Tools
Data Consistency
Performance
pt-table-checksum
● Connects to tables and checksums chunks of rows.
Monitoring ● Works across one table at a time
● Self aware
Features
● Slave aware
● Writes to table percona.checksums (by default)
© 2012 – Pythian
60. Eco-Tools
Data Consistency
Performance
pt-table-sync
● Synchronizes data (only) between tables
Monitoring ● Changes data!
● Runs no-op queries on the master to replicate to slaves
Features
● Foreign Key Funkiness
● Strongest with tables using PK or Unique key
© 2012 – Pythian
61. Eco-Tools
Data Consistency
Summary #1
Performance
Problem:
Monitoring
● Data drift detected
Solution:
Features
● Read Only Slaves
● Respect the binary logs
Workaround:
● Find and fix with pt-table tools
© 2012 – Pythian
66. Eco-Tools
Data Consistency
Meet the fortune tellers
Performance Replication Booster by Yoshinori Matsunobu
Monitoring faker by MySQLatFacebook
Features
mk-slave-prefetch by maatkit
slave readahead by Anders Karlsson
© 2012 – Pythian
67. Eco-Tools
Data Consistency
Meet the fortune tellers
Performance Replication Booster by Yoshinori Matsunobu
Monitoring faker by MySQLatFacebook
Features
mk-slave-prefetch by maatkit
slave readahead by Anders Karlsson
© 2012 – Pythian
68. Eco-Tools
Data Consistency
Meet the fortune tellers
Performance Replication Booster by Yoshinori Matsunobu
Monitoring faker by MySQLatFacebook
Features
mk-slave-prefetch by maatkit
slave readahead by Anders Karlsson
© 2012 – Pythian
69. Eco-Tools
Data Consistency
Meet the fortune tellers
Performance Replication Booster by Yoshinori Matsunobu
Monitoring faker by MySQLatFacebook
Features
mk-slave-prefetch by maatkit
slave readahead by Anders Karlsson
© 2012 – Pythian
70. Eco-Tools
Data Consistency
Meet the fortune tellers
Performance Replication Booster by Yoshinori Matsunobu
Monitoring faker by MySQLatFacebook
Features
mk-slave-prefetch by maatkit
slavereadahead by Anders Karlsson
© 2012 – Pythian
71. Eco-Tools
Data Consistency
Summary #1
Performance
Problem:
Monitoring
● serialized application of relay log
events can cause slave lag.
Features Solution:
● multithreaded slaves
Workaround:
● Hardware hack
● Prefetch
© 2012 – Pythian
72. Eco-Tools
http://code.openark.org/forge/openark-kit
Data Consistency
oak-get-slave-lag
Reports if slave is lag exceeds given threshold
Performance
●
● Works off SHOW SLAVE STATUS
Monitoring ● Bound to native replication
● Can be used in load balancer to determine if slave should be accessed
Features
oak-show-replication-status
● Reports replication status of slaves connected to a master
● Non-recursive (doesn't show if slave is a master and it's connected
slaves)
● Also depends on slaves being connected at runtime
● Can be used in load balancer to determine if slave should be
accessed
© 2012 – Pythian
73. Eco-Tools
Data Consistency
--pt-heartbeat
Performance
● Monitors replication lag on MySQL or Postgres
Monitoring ● Operates beyond native replication (Tungsten,
etc)
Features
● Supports multi tiered replication topology
● Can Emulate GTID (Global Transaction ID)
© 2012 – Pythian
74. Eco-Tools
--pt-slave-delay
Data Consistency ● Intentionally causes slave to lag behind it's master
● Works by starting and stopping slave threads
Performance ● Helpful as a means to retrieve old/lost data
● Considered a good means to reproducing race
Monitoring condition to test application
Features
--pt-slave-restart
● Monitors replica(s) for errors
● Brute force method of restarting replication
● Not for regular consumption
● Does not 'fix' replication
© 2012 – Pythian
76. Multi-threaded slaves
• Like Tungsten Replicator, operations are only parallel across
multiple distinct databases.
• Requires:
--slave-parallel-workers
(+relay_log_info_repository='table';)
(+STOP SLAVE; START SLAVE;)
• Tunable:
--slave_pending_jobs_size_max
--slave_checkpoint_group
--slave_checkpoint_period
© 2012 – Pythian
77. Binary log group commit
• Improves performance by writing several events to the binary
log file as opposed to one.
• Not “new”, previously seen in MariaDB 5.3.
• Requires no extra config
• Tunable
--binlog_order_commits
--binlog_max_flush_queue_time
© 2012 – Pythian
78. Global Transaction Identifier
• GTID is combination of the server uuid and the transaction_id
which is simply a sequential id on that server.
• Requires:
–log-bin
–log-slave-updates
–gtid-mode=ON
–disable-gtid-unsafe-statements (becomes enforce-gtid-consistency)
• CHANGE MASTER TO ... MASTER_AUTO_POSITION=1
• Required for some of the new MySQL Utilities
(mysqlrpladmin and mysqlfailover)
• http://mysqlmusings.blogspot.co.uk/2012/10/round-robin-
replication-using-gtid.html
© 2012 – Pythian
79. Global Transaction Identifier
mysql> show slave statusG
*************************** 1. row ***************************
...
Master_Log_File: mysql_sandbox5608-bin.000003
Read_Master_Log_Pos: 1257
...
Relay_Master_Log_File: mysql_sandbox5608-bin.000003
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
...
Exec_Master_Log_Pos: 1257
...
Master_UUID: 2a499ad4-388b-11e2-ae3c-12313c035dd8
...
Retrieved_Gtid_Set: 2A499AD4-388B-11E2-AE3C-12313C035DD8:1-10
Executed_Gtid_Set: 2A499AD4-388B-11E2-AE3C-12313C035DD8:1-10,
2AFB091E-388B-11E2-AE3C-12313C035DD8:1 (inconsistent case fixed in 5.6.10)
mysql> show global variables like 'server_uuid';
+---------------+--------------------------------------+
| Variable_name | Value |
+---------------+--------------------------------------+
| server_uuid | 2afb091e-388b-11e2-ae3c-12313c035dd8 |
+---------------+--------------------------------------+
© 2012 – Pythian
80. Crash safe slaves
• master.info and relay-log.info files are updated immediately
after a transaction commits, however there's still a chance of a
crash between the commit and the files being updated.
• New variables: master_info_repository and
relay_log_info_repository allow positional information to be
stored in tables.
• By default the master_info_repository and
relay_log_info_repository system tables use InnoDB storage
engine and are located in the MySQL db.
© 2012 – Pythian
81. Replication Checksums
• Not equivalent to pt-table-checksum, these checks are focused
on the events written in the binary log and replication.
• Uses CRC-32 checksum to verify the events in the binary log
and relay logs.
• binlog_checksum generates checksum in the binlog
• master_verify_checksum verifies events when read from the
binary log
• slave_sql_verify_checksum verifies events when read from the
relay log
© 2012 – Pythian
82. Slave Delay
• Provides a means of enforcing slave lag, similar to pt-slave-delay.
• Requires:
CHANGE MASTER TO MASTER_DELAY = N;
mysql> show slave statusG
*************************** 1. row ***************************
...
SQL_Delay: 0
SQL_Remaining_Delay: NULL
...
© 2012 – Pythian
83. Remote Binlog Backup
• Uses mysqlbinlog to provide functionality to backup binary logs
without the need of a slave.
• Requires:
--read-from-remote-server (or -R)
--raw
• Optional:
--stop-never
--stop-never-slave-server-id=id:
--result-file
© 2012 – Pythian
84. 5.6 Replication Potpourri
• Optimized row-based replication
• Informational log events – useful new feature enables the
original statement to be shipped in the binary log along with the
corresponding row before / after image.
• Slave_last_heartbeat – pt-heartbeat it is not, heartbeat
configured in CHANGE MASTER TO ... however only kicks in
during periods of inactivity.
© 2012 – Pythian
87. MySQL Utilities
• MySQL Utilities can be installed via MySQL Workbench, or as a
standalone package.
• Set of scripts written in Python.
http://dev.mysql.com/doc/workbench/en/mysql-utilities.html
https://launchpad.net/mysql-utilities
© 2012 – Pythian
88. MySQL Utilities
• mysqldbcompare • mysqlrplcheck
• mysqldbcopy • mysqlrplshow
• mysqldbexport • mysqlserverclone
• mysqldbimport • mysqlserverinfo
• mysqldiff • mysqluserclone
• mysqldiskusage • Mysqluc (cmd line)
• mysqlfailover • mut (tests)
• mysqlindexcheck
• mysqlmetagrep
• mysqlprocgrep
• mysqlreplicate
• mysqlrpladmin
© 2012 – Pythian
89. MySQL Utilities
• mysqlfailover
Performs replication health monitoring, provides automatic
failover on replication topologies (GTID, MySQL Server 5.6.5+)
• mysqlreplicate
Automates the process of setting up replication
• mysqlrpladmin
Administers the replication topology, allows recovery of the
master, commands include elect, failover, gtid, health, start,
stop, and switchover.
• mysqlrplcheck
Check replication configuration, tests binary logging on master.
• mysqlrplshow
Show slaves attached to master, can search recursively, shows
the replication topology as a graph or list.
© 2012 – Pythian
90. Thank you and Q&A
To contact us…
sales@pythian.com
1-877-PYTHIAN
To follow us…
http://www.pythian.com/news/
http://www.facebook.com/pages/The-Pythian-Group/163902527671
@pythian
@pythianjobs
http://www.linkedin.com/company/pythian
© 2012 – Pythian
