In Python Data Analysis, we used various profiling tools. These tools mostly had to do with measuring execution times. However, memory is also important, especially if we don't have enough of it. Memory leaks are a common issue with computer programs that we can find by performing memory profiling. Leaks occur when we don't release memory that is not needed. Problems also may occur when we use data types that require more memory than we need, for instance, NumPy float64 arrays when integer arrays will do.

The Python memory_profiler module can profile memory usage of code line by line. Once you install it, you can also use the module in an IPython notebook via various magic commands. The module works by communicating with the operating system. On Windows, you will require the Python psutil package for communication.

$ pip install memory-profiler 

import numpy as np


def test_me():
    a = np.random.random((999, 99))
    b = np.random.random((99, 99))
    a.ravel()
    b.tolist()

How to do it...

1. The imports are as follows:

   import dautil as dl
   from mem_test import test_me

2. Load the IPython extension as follows:

   %load_ext memory_profiler

3. Profile the test script line-by-line with the following command:

   %mprun -f test_me test_me()

Refer to the following screenshot for the end result:

The code is in the profiling_memory.ipynb file in this book's code bundle.