In this chapter we will cover:
- Mirroring queues
- Synchronizing queues
- Optimizing mirror policies
- Distributing messages between a couple of brokers
- Creating a geographical cluster replication
- Filtering and forwarding messages
- Combining the high-availability technologies together
- Client high availability
RabbitMQ approaches the issue of high availability by replicating data, the same as storage solutions (think of the RAID solutions), databases, and all the IT infrastructures when data integrity and service continuity are of primary importance.
In fact, the objective of these kinds of solutions is not only to avoid the possibility of data loss but also to avoid any downtime due to both scheduled maintenance and system malfunctions.
We will see how to use the simple but effective solution provided by RabbitMQ with queue mirroring. Through the recipes, we will see different use cases and optimizations that can be approached to minimize the performance price that you always have to pay when dealing with high availability.
Then, we will see how to perform geographical replication. This is an approach needed when the required Quality of Service (QoS) is so high that the application needs to be available even when a whole site is down (for example, due to a problem on the grid and the emergency power supply, a natural disaster, or just human error).
This approach is also suggested when using cloud computing resources. For example, when using Amazon Web Services (AWS), it is highly recommended to distribute the application among different availability zones (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). Since the link among different zones has much higher latency, it would not be reasonable to create a cluster across different availability zones, but there should be a cluster per zone, synchronized as we will see in the Creating a geographical cluster replication recipe.
Whether it is a simple, mirrored queue or a geographically replicated cluster, clients can assume to have successfully delivered and secured their messages to the mirrored queues only after they receive the corresponding notifications back (with ack messages using the publisher-confirms RabbitMQ extensions, or by using transactions that are of AMQP standard).
Note
In case you are dealing with a geographically replicated cluster, after a client publishes a message, RabbitMQ sends the ack message back to the client after the corresponding message has been replicated across the nodes, mirroring the queues in the local cluster. The geographical replication of the message is asynchronous instead. The message published on the local cluster is already to be considered secured and usually, the asynchronous copy to remote clusters is tolerable for disaster-recovery situations.
In the Client high availability recipe, we will see how to correctly implement a client that will be able to deal with the conditions that are common when connecting to high-availability RabbitMQ clusters.