The following table provides a quick comparison to some of the more salient comparison points between the four to five Linux I/O technologies we have seen, namely: the blocking read(2)/write(2) (and the SG-I/O/positioned [p]read[v](2)/[p]write[v](2)), memory mapping, non-blocking (mostly synchronous) DIO, and non-blocking asynchronous AIO:
| I/O Type | APIs | Pros | Cons |
|
Blocking |
[p]read[v](2) /[p]write[v](2) | Easy to use | Slow; double-copy of data buffers |
| Memory Mapping | mmap(2) | (Relatively) easy to use; fast (in memory I/O); single copy of data (a zero-copy technique); works best when RAM:Storage :: ~ 1:1 |
MMU-intensive (high page table overhead, meta-work) when RAM: Storage ratio is 1:N (N>>1) |
| DIO (non-blocking, mostly synchronous) |
open(2) with O_DIRECT flag | Zero-copy technique; no impact on page cache; control over caching; some control over I/O scheduling | Moderately complex to set up and use: app must perform its own caching |
| AIO (non-blocking, asynchronous) |
<Various: see aio(7) - POSIX AIO, Linux io_*(2), and so on> | Truly async and non-blocking—required for high-performance apps; zero-copy technique; no impact on page cache; full control over caching, I/O and thread scheduling | Complex to set up and use |
In the Further reading section on the GitHub repository, we provide links to two blog articles (from two real-world products: Scylla, a modern high-performance distributed No SQL data store, and NGINX, a modern high-performance web server), that discuss in depth how these alternative powerful I/O technologies (AIO, thread pools) are used in (their respective) real-world products; do take a look.