Insufficient Memory (Reserved Virtual Memory)

Allocated dynamic memory such as process heap can remain reserved after deallocation and its virtual memory region might become unavailable for usage. One example of this we encountered while debugging a .NET service. During peak usage it reported various out-of-memory events but its managed heap was healthy and didn't consume much. However, its process heap statistics showed a large reserved heap segment missing in a similar memory dump from a development environment. Remaining allocated entries in that heap segment contained a specific module hint (Volume 6, page 92) that allowed us to suggest removing a 3rd-party product from a production environment.

In order to provide a proof of that possible scenario of reserved heap regions we created a special modeling application:

int _tmain(int argc, _TCHAR* argv[])
{
    static char *pAlloc[1000000];
     for (int i = 0; i < 1000000; i++)
     {
        pAlloc[i] = (char *)malloc (1000);
     }
     getc(stdin);
     for (int i = 0; i < 1000000; i++)
     {
        free(pAlloc[i]);
     }
     getc(stdin);
     return 0;
}

Here's the debugging log:

0:001> .symfix c:mss

0:001> .reload
Reloading current modules
.....

After allocation:

0:001> !heap -s
LFH Key : 0x156356e0
Termination on corruption : ENABLED
Heap      Flags   Reserv  Commit  Virt    Free   List  UCR Virt Lock Fast
(k)       (k)     (k)     (k)     length  blocks cont. heap
-------------------------------------------------------------------------
00520000 00000002 1024    112     1024    8      1     1    0   0     LFH
007e0000 00001002 1019328 1012444 1019328 131    68    67   0   0     LFH
-------------------------------------------------------------------------

0:001> g
(1588.14b0): Break instruction exception - code 80000003 (first chance)
eax=7efda000 ebx=00000000 ecx=00000000 edx=770ff85a esi=00000000
edi=00000000
eip=7707000c esp=00f0f7e4 ebp=00f0f810 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
ntdll!DbgBreakPoint:
7707000c cc int 3

After deallocation:

0:001> !heap -s
LFH Key : 0x156356e0
Termination on corruption : ENABLED
Heap     Flags    Reserv  Commit Virt    Free   List  UCR  Virt Lock Fast
(k)      (k)      (k)     (k)    length  blocks cont. heap
-------------------------------------------------------------------------
00520000 00000002 1024    112    1024    8      1     1    0 0 LFH
007e0000 00001002 1019328 73040  1019328 71365  419   165  0 0 LFH
External fragmentation 97 % (419 free blocks)
Virtual address fragmentation 92 % (165 uncommited ranges)
-------------------------------------------------------------------------

0:001> !address -summary
--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free                               26              3fbe7000 (1019.902 Mb)           49.80%
<unclassified>                     752             3f8ec000 (1016.922 Mb)   98.92%  49.66%
Image                              41              76b000 ( 7.418 Mb)       0.72%   0.36%
Stack                              6               200000 ( 2.000 Mb)       0.19%   0.10%
MemoryMappedFile                   8               1af000 ( 1.684 Mb)       0.16%   0.08%
TEB                                2               2000 ( 8.000 kb)         0.00%   0.00%
PEB                                1               1000 ( 4.000 kb)         0.00%   0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_PRIVATE                        734             3f8a2000 (1016.633 Mb)   98.89%   49.64%
MEM_IMAGE                          68              9b8000   ( 9.719 Mb)     0.95%    0.47%
MEM_MAPPED                         8               1af000   ( 1.684 Mb)     0.16%    0.08%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_FREE                           26              3fbe7000 (1019.902 Mb)           49.80%
MEM_RESERVE                        374             3f6e8000 (1014.906 Mb)   98.72%  49.56%
MEM_COMMIT                         436             d21000 ( 13.129 Mb)      1.28%   0.64%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal
PAGE_READWRITE                     383             725000 ( 7.145 Mb)       0.69%   0.35%
PAGE_EXECUTE_READ                  10              414000 ( 4.078 Mb)       0.40%   0.20%
PAGE_READONLY                      29              1cd000 ( 1.801 Mb)       0.18%   0.09%
PAGE_WRITECOPY                     10              12000  ( 72.000 kb)      0.01%   0.00%
PAGE_READWRITE|PAGE_GUARD          4               9000 ( 36.000 kb)        0.00%   0.00%

--- Largest Region by Usage ----------- Base Address -------- Region Size ----------
Free                                    3f0c0000              33050000 ( 816.313 Mb)
<unclassified>                          158a1000              fcf000 ( 15.809 Mb)
Image                                   1083000               3d1000 ( 3.816 Mb)
Stack                                   200000                fd000 (1012.000 kb)
MemoryMappedFile                        7efe5000              fb000 (1004.000 kb)
TEB                                     7efda000              1000 ( 4.000 kb)
PEB                                     7efde000              1000 ( 4.000 kb)

We see that free memory available for allocation was only 816 Mb.