Memory Management

  • Memory management in Python involves a private heap containing all Python objects and data structures.
  • The management of this private heap is ensured internally by the Python memory manager.
  • The Python memory manager has different components which deal with various dynamic storage management aspects, like sharing, segmentation, preallocation or caching.
  • It is important to understand that the management of the Python heap is performed by the interpreter itself and that the user has no control over it, even if they regularly manipulate object pointers to memory blocks inside that heap.
  • To avoid memory corruption, extension writers should never try to operate on Python objects with the functions exported by the C library: malloc(), calloc(), realloc() and free().

How Much Memory Python Objects Use

sys.getsizeof

use sys.getsizeof to measure.

Int

  • The same integer always has same address and same memory size, no matter it is variable or literal.
  • Memory size of Integer 1 is 28, and Memory size of 1000000000000000000000000000000 is 40.

Memory Profiler

  • To gauge and measure the actual memory usage of your program, you can use the memory_profiler module.

  • You decorate a function (could be the main function) with an @profiler decorator, and when the program exits, the memory profiler prints to standard output a handy report that shows the total and changes in memory for every line.

tracemalloc

  • tracemalloc is a Python module that acts as a debug tool to trace memory blocks allocated by Python.

Once tracemalloc is enabled, you can obtain the following information:

  • identify where the object was allocated
  • give statistics on allocated memory
  • detect memory leaks by comparing snapshots