Why Python Multithreading Struggles and When to Switch to Multiprocessing

Background

Recently I have been studying Python multithreading. Many experienced developers claim that “Python multithreading is a chicken‑rib, use multiprocessing instead”. This article explores why.

What is the GIL?

The Global Interpreter Lock (GIL) is a mechanism introduced in Python’s design to protect internal data structures. Only one thread can hold the GIL at a time, so on a single‑core CPU a multithreaded program is merely concurrent, not parallel.

How a Python thread runs

Acquire the GIL.

Execute bytecode until it reaches a blocking operation such as sleep or the interpreter decides to suspend it.

Release the GIL.

In CPython 2.x the GIL is released when the thread performs I/O or after a fixed number of “ticks” (default 100), which can be changed with sys.setcheckinterval. Each release causes lock contention and context‑switch overhead, and because only one thread can run at a time, multithreading does not scale on multi‑core CPUs.

When is Python multithreading useful?

We can distinguish two typical workloads:

CPU‑bound code (tight loops, heavy calculations). The frequent GIL releases and reacquisitions add overhead, so multithreading offers little benefit.

I/O‑bound code (file handling, network crawling). While one thread waits for I/O, another can run, reducing idle CPU time and improving overall throughput.

In Python 3.x the GIL release policy changed from tick‑based to time‑based, which slightly improves CPU‑bound scenarios but does not eliminate the fundamental limitation that only one thread executes Python bytecode at a time.

Why multiprocessing often wins

Each process has its own independent GIL, allowing true parallel execution on multi‑core machines. Therefore, for CPU‑intensive tasks on multi‑core CPUs, using the multiprocessing module generally yields higher performance than threading.

Conclusion

Python multithreading is suitable mainly for I/O‑bound workloads. For CPU‑bound workloads on multi‑core systems, spawning multiple processes is the more effective way to achieve parallelism and improve execution efficiency.