OpenStack can be backed by a number of database backends, and one of the most common options is MySQL or its other open source fork, MariaDB. There are a number of ways to make MariaDB more resilient and highly available. The following approach uses a Load Balancer to front a multi-read/write master with Galera, taking care of the synchronous replication required in such a setup. Galera is a synchronous multimaster cluster for MariaDB InnoDB databases. Galera clusters allow synchronous data writes across all nodes with any node being able to take that write in a fully active/active topology. It features automatic node management- that is, failed nodes are removed from the cluster and new nodes are automatically registered. The advantage of this is that we are adding resilience in the event of a database node failure, as each node stores a copy of the data. Galera clusters consist of odd-numbered nodes. This is important when a node fails. Galera takes a quorum vote from the remaining nodes to determine the state. Quorum requires a majority, that is, you cannot have automatic failover in a two-node cluster. This is because the failure of one causes the remaining nodes to automatically go into a nonprimary state. Clusters that have an even number of nodes risk split-brain conditions. If should you lose network connectivity somewhere between the partitions in a way that causes the number of nodes to split exactly in half, neither of the partitions can retain quorum and both enter a nonprimary state.
Ensure that you have three servers running Ubuntu 14.04 and at least one interface that will be used to access the machines and be configured for Galera replication. Follow the instructions at https://github.com/OpenStackCookbook/MariaDB-Galera to bring up a suitable vagrant environment. These steps are also repeated in the following How to do it… section.
For this recipe, we will install MariaDB and Galera on three nodes that we will call Galera1, Galera2, and Galera3. They will each have a single IP assigned to them on the 172.16.0.0/16 network and the interface will be referenced as eth1 on the machines. The IPs used in the following steps are 172.16.0.191, 172.16.0.192, and 172.16.0.193. They will all be running Ubuntu 14.04 LTS. The steps are as follows:
- On the first server,
Galera1, we configureaptto be able to retrieve MariaDB Cluster 10.0 packages with the following command:apt-get install software-properties-common sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xcbcb082a1bb943db sudo add-apt-repository 'deb http://lon1.mirrors.digitalocean.com/mariadb/repo/10.0/ubuntu trusty main' sudo apt-get update
- Install the packages by running the following command:
DEBIAN_FRONTEND=noninteractive apt-get install sync galera mariadb-galera-server - Ensure that MariaDB isn't running by executing the following command:
sudo service mysql stop - Repeat step 1 to step 3 for
Galera2andGalera3. - We then configure MariaDB to use Galera by editing the
/etc/mysql/conf.d/galera.cnffile to include the following contents:[mysqld] # mysql settings binlog_format=ROW default-storage-engine=innodb innodb_autoinc_lock_mode=2 query_cache_size=0 query_cache_type=0 bind-address=0.0.0.0 # galera settings wsrep_provider=/usr/lib/galera/libgalera_smm.so wsrep_cluster_name="my_wsrep_cluster" wsrep_cluster_address="gcomm://172.16.0.191,172.16.0.192,172.16.0.193" wsrep_sst_method=rsync
- Log in to
Galera1and execute the following command to start up a new Galera replication cluster:sudo service mysql start --wsrep-new-cluster - Log in to
Galera2andGalera3to execute the following command on each node:sudo service mysql start - Test whether
mysql servicewas started by logging into any of the nodes and running the following command:mysql -u root -e 'SELECT VARIABLE_VALUE as "cluster size" FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME="wsrep_cluster_size"'
It should return the following output:
We configured three servers running Ubuntu 14.04 to be able to install MariaDB and Galera. We do this by adding the MariaDB repository to our apt environment and installing the appropriate packages.
Once this has been done on each of the three nodes, we configure MariaDB to use Galera by editing the /etc/mysql/conf.d/galera.cnf file. This has a section that describes how Galera is used. We specify a cluster name, the IP addresses used in the cluster, and the method by which the data will be replicated:
wsrep_provider=/usr/lib/galera/libgalera_smm.so wsrep_cluster_name="my_wsrep_cluster" wsrep_cluster_address="gcomm://172.16.0.191,172.16.0.192,172.16.0.193" wsrep_sst_method=rsync
After this, we start up the cluster. To do this, we choose one of the nodes and execute the following command:
service mysql start --wsrep-new-cluster
We then start up the remaining nodes as usual using the following command:
service mysql start