There are quite a few settings that may be tweaked for OSDs; optimal values vary depending on the speeds of your OSD and journal drives, your workload, and the number of CPU cores available. The suggestions here must be researched and tested in the context of your unique deployment:

[osd]
filestore_queue_max_bytes = 1048576000
filestore_queue_max_ops = 50000

The above settings work well for the authors to enhance the performance of FileStore OSDs deployed on fast SSDs. Your mileage may vary, especially on HDD OSDs:

[osd]
osd_op_threads = 16
osd_disk_threads = 4

These thread settings are sometimes controversial; some hold that increasing them can benefit performance, others that they increase contention and can lead to thrashing. Their effects vary by Ceph release, internal management of work queues has evolved. Kernel versions and I/O scheduler settings also affect what's best here. Tweak these carefully and benchmark before and after. If your systems have a larger number of OSD daemons but a small number of CPU cores, you may experience context switching overhead by setting them too high.

We discussed scrub settings in Chapter 19, Operations and Maintenance as well; you may wish to consider lengthening your osd_deep_scrub_interval to minimize contention, especially on HDD systems.