Removing a Ceph MON is generally not a very frequently required task. When you remove monitors from a cluster, consider that Ceph monitors use the PAXOS algorithm to establish consensus about the master cluster map. You must have a sufficient number of monitors to establish a quorum for consensus on the cluster map. In this recipe, we will learn how to remove the ceph-node4 monitor from the Ceph cluster. At the current time of writing this book, ceph-ansible does not support scaling down of Ceph MON nodes in a cluster and this must be done manually.

1. Check the monitor status:

# ceph mon stat

2. Stop the monitor service on ceph-node4:

root@ceph-node1 # systemctl -H ceph-node4 stop ceph-mon.target

3. Remove the monitor from the cluster:  

# ceph mon remove ceph-node4

4. Check to see that your monitors have left the quorum:

# ceph quorum_status --format json-pretty

5. Update the /etc/ansible/hosts file and remove ceph-node4 from the [mons] section so ceph-node4 is not redeployed as a mon and the ceph.conf file is properly updated:
   
6. You can choose to back up the monitor data on ceph-node4 or remove it. To back it up, you can create a removed directory and move the data there:

 # mkdir /var/lib/ceph/mon/removed
 # mv /var/lib/ceph/mon/ceph-ceph-node4 /var/lib/ceph/mon/removed/ceph-ceph-node4

7. If you choose not to back up the monitor data, then remove the monitor data on ceph-node4:

# rm -r /var/lib/ceph/mon/ceph-ceph-node4

8. Re-run the Ansible playbook to update the ceph.conf on all the nodes in the cluster to complete the removal of monitor ceph-node4:

root@ceph-node1 # ansible-playbook site.yml

9. Finally, check the monitor status; the cluster should have three monitors: