How to Slash Python Memory Usage with __slots__ and Size Inspection

Problem Overview

In a project that required storing and processing a large dynamic list, testers complained about insufficient memory. The article shares a simple method—adding a single line of code—to address this issue.

Initial Class Definition

class DataItem(object):
    def __init__(self, name, age, address):
        self.name = name
        self.age = age
        self.address = address

Measuring Object Size

Using sys.getsizeof on two instances shows both report 56 bytes, which is misleading because Python objects carry additional hidden overhead.

d1 = DataItem("Alex", 42, "-")
print("sys.getsizeof(d1):", sys.getsizeof(d1))

d2 = DataItem("Boris", 24, "In the middle of nowhere")
print("sys.getsizeof(d2):", sys.getsizeof(d2))

Even an empty string occupies 33 bytes and an integer 24 bytes, illustrating the hidden cost of dynamic typing.

Recursive Size Calculation

def get_size(obj, seen=None):
    # Recursively finds size of objects
    size = sys.getsizeof(obj)
    if seen is None:
        seen = set()
    obj_id = id(obj)
    if obj_id in seen:
        return 0
    seen.add(obj_id)
    if isinstance(obj, dict):
        size += sum(get_size(v, seen) for v in obj.values())
        size += sum(get_size(k, seen) for k in obj.keys())
    elif hasattr(obj, '__dict__'):
        size += get_size(obj.__dict__, seen)
    elif hasattr(obj, '__iter__') and not isinstance(obj, (str, bytes, bytearray)):
        size += sum(get_size(i, seen) for i in obj)
    return size

Applying this function yields realistic memory usage: 460 bytes for d1 and 484 bytes for d2. When placed in a list, the sizes reflect shared references, e.g., get_size([d1]) returns 532 bytes and get_size([d1, d2]) returns 863 bytes.

Reducing Memory with __slots__

By defining __slots__, Python skips the per‑instance __dict__, drastically cutting memory consumption.

class DataItem(object):
    __slots__ = ['name', 'age', 'address']
    def __init__(self, name, age, address):
        self.name = name
        self.age = age
        self.address = address

After this change, get_size(d1) drops to about 64 bytes—a seven‑fold reduction—while object creation becomes roughly 20 % faster.

Large‑Scale Test

data = []
for p in range(100000):
    data.append(DataItem("Alex", 42, "middle of nowhere"))
import tracemalloc
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
total = sum(stat.size for stat in top_stats)
print("Total allocated size: %.1f MB" % (total / (1024*1024)))

Without __slots__ the allocation is ~16.8 MB; with __slots__ it drops to ~6.9 MB.

Trade‑offs

Using __slots__ disables the instance __dict__, which means typical JSON serialization via json.dumps(self.__dict__) no longer works. A workaround is to build a dictionary from __slots__ manually:

def toJSON(self):
    data = {}
    for var in self.__slots__:
        data[var] = getattr(self, var)
    return json.dumps(data)

Dynamic addition of new attributes is also prevented, but this is acceptable for many use cases.

Conclusion

While Python remains a convenient and readable language, its default object model can be memory‑inefficient. Employing __slots__ or using libraries like NumPy for C‑style structures can provide substantial memory savings when performance is critical.