Busy XFS-based FileStore OSDs may find that these settings optimize performance, especially if you have the luxury of using SSDs. The XFS metadata sync operations have traditionally been intrusive and somewhat painful. When the sync thread awakens it may pre-empt the writes of all other processes (threads), which can interfere with Ceph's journal and data flushes. The fs.xfs settings here help to mitigate that effect.

The fs.aio-max-nr setting expands the queue for disk operations. Modest to moderate OSD nodes say those with fewer than 20 OSDs, may do fine with the default value. As the per-host OSD density increases, it increasingly becomes a limiting factor. Throughput benefits from scaling this value proportionally to drive count; this value of 50 million should be ample for even the densest Ceph OSD nodes:

fs.xfs.xfssyncd_centisecs=720000
fs.xfs.xfsbufd_centisecs=3000       # pre-4.0 kernels only
fs.xfs.age_buffer_centisecs=720000  # pre-4.0 kernels only
fs.aio-max-nr=50000000