Before any data can be mirrored from the Ceph cluster to the backup cluster we first need install the rbd-mirror daemon on the backup cluster, enable mirroring on the data pool and then add a peer cluster to the pool:

1. On ceph-node5 in the backup cluster install and configure the rbd-mirror. The client ID of remote (our user) is what the rbd-mirror daemon will use:

        root@ceph-node5 # yum install -y rbd-mirror
                        # systemctl enable ceph-rbd-mirror.target
                        # systemctl enable ceph-rbd-mirror@remote
                        # systemctl start ceph-rbd-mirror@remote

2. Enable mirroring of the whole pool named data in cluster ceph. The syntax is as follows:

                        # rbd mirror pool enable <pool> <mode>
        root@ceph-node1 # rbd mirror pool enable data pool

3. Enable mirroring of the whole pool named rbd in the cluster backup:

        root@ceph-node5 # rbd mirror pool enable data pool

4. For the rbd-mirror daemon to discover it's peer cluster, we now must register the peer to the pool. We will need to add the ceph cluster as a peer to the backup cluster. The syntax is as follows:

                       # rbd mirror pool peer add <pool> 
                         <client-name@cluster-name>
       root@ceph-node5 # rbd mirror pool peer add 
                         data client.local@ceph

5. Next, we will validate the peer relationship between the pools and the cluster. The syntax is as follows:

                        # rbd mirror pool info <pool>
        root@ceph-node5 # rbd mirror pool info rbd

Mirroring is now enabled at the pool level for pool data in the ceph and backup clusters and a pool peer is configured for the data pool in the cluster backup.

6. Review the data pool in each cluster and see there are currently no RBD images in either site. Once verified we will create three new RBD images in the data pool with exclusive-lock and journaling enabled and watch them sync to the secondary backup cluster:

        # rbd ls data
        # rbd create image-1 --size 1024 --pool data 
          --image-feature exclusive-lock,journaling

1. 1. Pool mirrors can be polled for a status of the images as they sync to the backup cluster:

               # rbd mirror pool status data

2. 2. Viewing the remote site for before and after on the data pool:

3. 3. Viewing the image sync status on the remote site:

4. 4. Viewing the healthy state of the journal replaying the image on the remote site:

Replaying is the image state we want to see as this means the rbd-mirror daemon sees the images in sync and is replaying the journals for any changes to the images that need to be sync.

7. We will now delete the three images from the ceph cluster and watch as they are removed from the backup cluster:

        # rbd rm -p data image-<num>

8. Viewing the pool status to validate the images are removed from the remote site:

        # rbd mirror pool status data

If at any point and time you choose to disable mirroring on a pool this can be done via the rbd mirror peer remove and rbd mirror pool disable commands for the chosen pool. Please note when mirroring is disabled for a pool you also disable mirroring on any images residing in the pool.

Following is the syntax for removing mirroring:

1. 1. To remove peer:

              # rbd mirror peer remove <pool-name> <peer-uuid>

2. 2. To remove mirroring on pool:

              # rbd mirror pool disable <pool>