How malloc(3) really behaves

The general consensus it that malloc(3) (and calloc(3) and realloc[array](3)) obtains its memory from the heap segment. This is indeed the case, but digging a bit deeper reveals that it's not always the case. The modern glibc malloc(3) engine uses some subtle strategies to make the most optimal use of available memory regions and the process VAS—which, especially on today's 32-bit systems, is fast becoming a rather scarce resource.

So, how does it work? The library uses a predefined MMAP_THRESHOLD variable  its value is 128 KB by default  to determine from where memory gets allocated. Let's imagine we are allocating n bytes of memory with malloc(n):

  • If n < MMAP_THRESHOLD, use the heap segment to allocate the requested n bytes

  • If n >= MMAP_THRESHOLD, and if n bytes are not available on the heap's free list, use an arbitrary free region of virtual address space to satisfy the requested n bytes allocation

How exactly is the memory allocated in the second case? Ah, malloc(3) internally calls mmap(2) – the memory map system call. The mmap system call is very versatile. In this case, it is made to reserve a free region of n bytes of the calling process's virtual address space!

Why use mmap(2)? The key reason is that mmap-ed memory can always be freed up (released back to the system) in an independent fashion whenever required; this is certainly not always the case with free(3).

Of course, there are some downsides: mmap allocations can be expensive because, the memory is page-aligned (and could thus be wasteful), and the kernel zeroes out the memory region (this hurts performance).

The mallopt(3) man page (circa December 2016) also notes that nowadays, glibc uses a dynamic mmap threshold; initially, the value is the usual 128 KB, but if a large memory chunk between the current threshold and DEFAULT_MMAP_THRESHOLD_MAX is freed, the threshold is increased to become the size of the freed block.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset