malloc(3) – some FAQs

The following are some FAQs that will help us to learn more about malloc(3):

  • FAQ 1 : How much memory can malloc(3) allocate with a single call?

A rather pointless question in practical terms, but one that is often asked!

The parameter to malloc(3) is an integer value of the size_t data type, so, logically, the maximum number we can pass as a parameter to malloc(3) is the maximum value a size_t can take on the platform. Practically speaking, on a 64-bit Linux, size_t will be 8 bytes, which of course, in bits is 8*8 = 64. Therefore, the maximum amount of memory that can be allocated in a single malloc(3) call is 2^64!

So, how much is it? Let's be empirical (it's important to read in Chapter 19, Troubleshooting and Best Practices, and the brief discussion there on The empirical approach).and actually try it out (note that the following code snippet has to be linked with the math library using the -lm switch):

    int szt = sizeof(size_t);
float max=0;
max = pow(2, szt*8);
printf("sizeof size_t = %u; "
"max value of the param to malloc = %.0f ",
szt, max);

The output, on an x86_64:

sizeof size_t = 8; max param to malloc = 18446744073709551616

Aha! That's a mighty large number; more readably, it's as follows:

2^64 = 18,446,744,073,709,551,616 = 0xffffffffffffffff

That's 16 EB (exabytes, which is 16,384 PB, which is 16 million TB)!

So, on a 64-bit OS, malloc(3) can allocate a maximum of 16 EB in a single call. In theory.

As usual, there's more to it: please see FAQ 2; it will reveal that the theoretical answer to this question is 8 exabytes (8 EB).

In practice, obviously, this would be impossible because, of course, that's the entire usermode VAS of the process itself. In reality, the amount of memory that can be allocated is limited by the amount of free memory contiguously available on the heap. Actually, there's more to it. As we shall soon learn (in the How malloc(3) really behaves section), memory for malloc(3) can come from other regions of the VAS, too. Don't forget there's a resource limit on data segment size; the default is usually unlimited, which as we discussed in this chapter, really means that there's no artificial limit imposed by the OS.

So, in practice, it's best to be sensible, not assume anything, and check the return value for NULL.

As an aside, what's the maximum value a size_t can take on a 32-bit OS? Accordingly, we compile on x86_64 for 32-bit by passing the -m32 switch to the compiler:

$ gcc -m32 mallocmax.c -o mallocmax32 -Wall -lm
$ ./mallocmax32
*** max_malloc() ***
sizeof size_t = 4; max value of the param to malloc = 4294967296
[...]
$

Clearly, it's 4 GB (gigabytes)  again, the entire VAS of a 32-bit process.

  • FAQ 2: What if I pass malloc(3) a negative argument?

The data type of the parameter to malloc(3), size_t, is an unsigned integer quantity  it cannot be negative. But, humans are imperfect, and Integer OverFlow (IOF) bugs do exist! You can imagine a scenario where a program attempts to calculate the number of bytes to allocate, like this:

num = qa * qb;

What if num is declared as a signed integer variable and qa and qb are large enough that the result of the multiplication operation causes an overflow? The num result will then wrap around and become negative! malloc(3) should fail, of course. But hang on: if the num variable is declared as size_t (which should be the case), the negative quantity will turn into some positive quantity!

The mallocmax program has a test case for this.

Here is the output when run on an x86_64 Linux box:

*** negative_malloc() ***
size_t max = 18446744073709551616
ld_num2alloc = -288225969623711744
szt_num2alloc = 18158518104085839872
1. long int used: malloc(-288225969623711744) returns (nil)
2. size_t used: malloc(18158518104085839872) returns (nil)
3. short int used: malloc(6144) returns 0x136b670
4. short int used: malloc(-4096) returns (nil)
5. size_t used: malloc(18446744073709547520) returns (nil)

Here are the relevant variable declarations:

const size_t onePB    = 1125899907000000; /* 1 petabyte */
int qa = 28*1000000;
long int ld_num2alloc = qa * onePB;
size_t szt_num2alloc = qa * onePB;
short int sd_num2alloc;

Now, let's try it with a 32-bit version of the program.

Note that on a default-install Ubuntu Linux box, the 32-bit compile may fail (with an error such as fatal error: bits/libc-header-start.h: No such file or directory). Don't panic: this usually implies that the compiler support for building 32-bit binaries isn't present by default. To get it (as mentioned in the Hardware-Software List document), install the multilib compiler package: sudo apt-get install gcc-multilib.

Compile it for 32-bit and run it:

$ ./mallocmax32 
*** max_malloc() ***
sizeof size_t = 4; max param to malloc = 4294967296
*** negative_malloc() ***
size_t max = 4294967296
ld_num2alloc = 0
szt_num2alloc = 1106247680
1. long int used: malloc(-108445696) returns (nil)
2. size_t used: malloc(4186521600) returns (nil)
3. short int used: malloc(6144) returns 0x85d1570
4. short int used: malloc(-4096) returns (nil)
5. size_t used: malloc(4294963200) returns (nil)
$

To be fair, the compiler does warn us:

gcc -Wall   -c -o mallocmax.o mallocmax.c
mallocmax.c: In function ‘negative_malloc’:
mallocmax.c:87:6: warning: argument 1 value ‘18446744073709551615’ exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
ptr = malloc(-1UL);
~~~~^~~~~~~~~~~~~~
In file included from mallocmax.c:18:0:
/usr/include/stdlib.h:424:14: note: in a call to allocation function ‘malloc’ declared here
extern void *malloc (size_t __size) __THROW __attribute_malloc__ __wur;
^~~~~~
[...]

Interesting! The compiler answers our FAQ 1 question now:

[...] warning: argument 1 value ‘18446744073709551615’ exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=] [...]

The maximum value you can allocate as per the compiler seems to be 9223372036854775807.

Wow. A little calculator time reveals that this is 8192 PB = 8 EB! So, we must conclude that  the correct answer to the previous question: How much memory can malloc allocate with a single call? Answer: 8 exabytes. Again, in theory.

  • FAQ 3: What if I use malloc(0)?

Not much; depending on the implementation, malloc(3) will return NULL, or, a non-NULL pointer that can be passed to free. Of course, even if the pointer is non-NULL, there is no memory, so don't attempt to use it.

Let's try it out:

    void *ptr;
ptr = malloc(0);
free(ptr);

We compile and then run it via ltrace:

$ ltrace ./a.out 
malloc(0) = 0xf50260
free(0xf50260) = <void>
exit(0 <no return ...>
+++ exited (status 0) +++
$

Here, malloc(0) did indeed return a non-NULL pointer.

  • FAQ 4: What if I use malloc(2048) and attempt to read/write beyond 2,048 bytes?

This is a bug of course  an out-of-bounds memory-access bug, further defined as a read or write buffer overflow. Hang on please, the detailed discussion of memory bugs (and subsequently, how to find and fix them) is the subject of Chapter 5, Linux Memory Issues, and Chapter 6Debugging Tools for Memory Issues.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset