RBD mirroring is an asynchronous replication of RBD images between multiple Ceph clusters. RBD mirroring validates a point-in-time consistent replica of any change to an RBD image, including snapshots, clones, read and write IOPS and block device resizing. RBD mirroring can run in an active+active setup or an active+passive setup. RBD mirroring utilizes the RBD journaling and exclusive lock features which enables the RBD image to record all changes to the image in order of which they occur. These features validate that a crash-consistent copy of the remote image is available locally. Before mirroring can be enabled on a Ceph cluster the journaling feature must be enabled on the RBD image.

The daemon responsible for ensuring point-in-time consistency from one Ceph cluster to the other is the rbd-mirror daemon. Depending on your chosen type of replication the rbd-mirror daemon runs on either a single cluster or an each participating in the mirroring:

• One way replication: Data is mirrored from a primary site to secondary site. RBD mirror runs only on secondary site.
• Two-way replication: Active-Active configuration. Data is mirrored from a primary site to secondary site and mirrored back from a primary site to secondary site. RBD mirror runs on both primary and secondary sites.

Since we store the RBD images on both local and remote pools it is best practices to ensure the CRUSH hierarchy backing the mirrored pool is the same speed, size and type media. Also, proper bandwidth should be allocated between sites to handle the mirroring traffic.