Database Replication in MySQL

By Ian Gilfillan

An introduction to replication
Recently, while having the knots pounded out of my body during a particularly
painful shiatsu lesson, I reflected on what put them there in the first place. Yes, 'the
database' was once more to blame. A busy database I work with saw one of its
tables jump from 3GB to 7GB overnight, as we imported archive data. As expected,
this had some performance impact. Unfortunately, I had not expected quite the
knock, and it turned out that this single database server could no longer handle the
load. No matter how much more I tried to optimize the queries, tweak the variables
or bump up the query cache, it was not enough. The machine could not take any
more memory, and a hardware upgrade would do little good (at least with the kind
of budget I have to play with). However, MySQL does not claim to be enterprise-
ready for nothing, and Yahoo and other high-volume users of MySQL certainly do not
run on one database server. There are a number of techniques to handle high
volumes, one of which I will introduce this month - MySQL replication (I will look at
others in future articles).

Replication allows you to take one database, make an exact copy of it on another
server, and set one of them (the slave) to take all its updates from the other (the
master). The slave reads the master's binary logs, which store all statements that
change a database, and repeats these on its database, keeping the two in exact
sync. Since a replicating database simply repeats statements, the databases are not
necessarily exactly in sync, and advanced users can take advantage of this. That is a
topic for another article however, and we will look at simple replication this month -
getting one database to be an exact copy of another one.

What replication is not

• Replication is not a backup policy. A mistyped DELETE statement will be

replicated on the slave too, and you could end up with two, perfectly
synchronized, empty databases. Replication can help protect against
hardware failure though.
• Replication is not an answer to all performance problems. Although updates
on the slave are more optimized than if you ran the updates normally, if you
use MyISAM tables, table-locking will still occur, and databases under high-
load could still struggle.
• Replication is not a guarantee that the slave will be in sync with the master at
any one point in time. Even assuming the connection is always up, a busy
slave may not yet have caught up with the master, so you can't simply
interchange SELECT queries across master and slave servers.

How to start replicating - the master server

 • Grant the slave permission to replicate with the REPLICATION SLAVE
privilege, for example as follows: GRANT REPLICATION SLAVE ON *.* TO
slave_user IDENTIFIED BY 'slave_password'
• If the master is not using the binary update log, add the following lines to the
my.cnf or my.ini configuration file, and restart the server:
•
• log-bin
• server-id=1

By convention, the master is usually server-id 1, and any slaves from 2

onwards, though you can change this if you wish. If the master is already
using the binary update log, either take note of the offset at the moment of
the backup (the next step), or use the RESET MASTER statement to clear all
binary logs and immediately begin the backup. You may want to make a copy
of the binary logs before doing this, in case you need to use the binary logs to
restore from backup.

• Make a backup of the database. You will use this to start the slave server.
Note the comments about the binary log above. You can also skip this step if
you use the LOAD DATA FROM MASTER statement, but see the comments
about locking the master below first.

How to start replicating - the slave server

• Add the following to the configuration file on the slave:

• master-host=master-hostname
• master-user=slave-user
• master-password=slave-password
• server-id=2
•
•

The slave user and slave password are those to which you set when you
granted REPLICATION SLAVE permission on the master. The server-id must
be a unique number, different to the master or any other slaves in the
system. There are also two other options: master-port, used if the master is
running on a non-standard port (3306 is default), and master-connect-retry,
a time in seconds for the slave to attempt to reconnect if the master goes
down. 60 seconds is default.

• Restore the data from the master, either as you would normally restore a
backup, or with the statement LOAD DATA FROM MASTER. The latter will lock
the master for the duration of the operation, which could be quite lengthy, so
you may not be able to spare the downtime.

Replication in action
Once the slave has started, replication should begin. Besides the obvious SELECT
queries, you can make sure this is working correctly with the following statements:

mysql> SHOW SLAVE STATUS\G

*************************** 1. row ***************************
Master_Host: masterhostname.co.za
Master_User: slave_server
Master_Port: 3306
Connect_retry: 60
Master_Log_File: master-bin.054
Read_Master_Log_Pos: 16664104
Relay_Log_File: slave-relay-bin.045
Relay_Log_Pos: 17657643
Relay_Master_Log_File: master-bin.054
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_do_db: vne
Replicate_ignore_db:
Last_errno: 0
Last_error:
Skip_counter: 0
Exec_master_log_pos: 16664104
Relay_log_space: 17657643

This is a mature slave that has been running a while. The master is already on the
54th binary log. You can see if the slave is running correctly by looking at the
Slave_IO_Running and Slave_SQL_Running. The most important field is the
Last_error field.

mysql> SHOW MASTER STATUS;

+----------------+----------+--------------+------------------+
| File | Position | Binlog_do_db | Binlog_ignore_db |
+----------------+----------+--------------+------------------+
| master-bin.054 | 16664104 | | |
+----------------+----------+--------------+------------------+

The above is from a master that has been running a while. It is already on binlog 54.

Starting to replicate from a particular point in the binary

logs
If you need to force the slave to begin at a certain point, usually when the master
has been running with an active binary log, you can do so as follows. The following
starts with the 3rd binary log, as position 420. You can find the position using
mysqlbinlog.

mysql> CHANGE MASTER TO MASTER_LOG_FILE='master-bin.003'

MASTER_LOG_POS=420;
Query OK, 0 rows affected (0.00 sec)
mysql> SLAVE START
The SLAVE START and SLAVE STOP commands are used to manually stop and start
the slave. The slave will also always stop if it comes across an error while replicating.

Removing old binary logs

On active databases, the binary logs tend to grow quite quickly. You may have used
RESET MASTER in the past to clear them, but you cannot do this to the master while
replicating! The statement to use is PURGE MASTER LOGS. First, make sure that all
slaves have replicated to at least the log beyond which you want to remove. For
example, in our earlier example, both the slave and the master are on log master-
bin.054, so we can safely remove master-log.053 and before, as follows:

mysql> PURGE MASTER LOGS TO 'master-bin.053';

MySQL will not allow you to remove a log that the master is still using though.

Replicating specific databases only

As mentioned earlier, the master and slave database server do not need to be
entirely in sync, with all databases and all tables from the master replicated onto the
slave. By default, the slave will replicate everything, but you can change this
behavior with the following options in the slave configuration file:

replicate-do-db=db_name (replicate this database)

replicate-ignore-db=db_name (don't replicate this database)
replicate-do-table=db_name.table_name (replicate this table)
replicate-ignore-table=db_name.table_name (don't replicate this table)
replicate--wild-do-table=db_name.table_name
(allows wildcards, e.g db% would be all databases beginning
with db)
replicate-wild-ignore-table=db_name.table_name (ignore all specified
tables, with wildcards)

These options can all be used multiple times in a single configuration. A couple of
other useful options:

replicate-rewrite-db=master_db->slave_db (allows you to use map

databases
that use different database names on each server)
log-slave-update (writes replicated statements to the slaves binary
logs)

A few complexities

• Before shutting down a slave server (mysqladmin-shutdown, not STOP

SLAVE, make sure it has no temporary tables open (these may be needed for
a statement to be replicated). You can do this with SHOW STATUS to see the
value of Slave_open_temp_tables. This annoying feature should be fixed
soon, so please check the latest documentation.
 • The USER(), UUID(), LOAD_FILE() and CONNECTION_ID() (before MySQL
4.1.1) functions do not work reliably on the slave (they are replicated without
changes).
• Before MySQL 4.1.1, FLUSH, ANALYZE, OPTIMIZE, and REPAIR statements
are not replicated. This means that if you change permissions on the master
by editing the tables directly, you will need to manually FLUSH PRIVILEGES
on the slave too.
• Make sure the slave and the master are using the same character set.

Replication is not the salvation it seems to be at first glance, and will improve greatly
in future versions of MySQL, but it is a useful addition to a DBA's armory. I hope that
you will find replication easy to get going. Good luck!