Master Python CPU Profiling: cProfile, line_profiler, pprofile & vprof Explained

In this article we discuss tools for analyzing CPU usage in Python scripts. CPU profiling measures code performance to locate inefficiencies.

We will look at four main tools:

cProfile

line_profiler

pprofile

vprof

cProfile

cProfile is a deterministic profiler built into CPython and PyPy. It collects statistics such as call counts and execution time with low overhead.

Usage on CPython:

On PyPy:

The textual output shows functions like list.append being called many times. Visualizing with gprof2dot and Graphviz produces a call graph:

The graph lists function name, total time percentage, self‑time percentage, and call count.

Example rewritten script improves performance:

line_profiler

line_profiler provides line‑level timing using Cython, with modest overhead. It must be installed via pip and the target function decorated with @profile. It does not support PyPy.

Output highlights the loops calling list.append as the most time‑consuming.

pprofile

pprofile is a pure‑Python profiler inspired by line_profiler, supporting both CPython and PyPy and offering thread profiling. It is slower than cProfile (28× on CPython, 10× on PyPy) but provides detailed per‑line statistics.

Install with pip and run:

Output includes call counts, time per call, and cumulative metrics, again showing list.append loops dominate.

vprof

vprof is a Python profiler that generates interactive visualizations (CPU usage, code heatmaps, memory, etc.) via a Node.js web interface.

Install with pip and run on CPython or PyPy to see code heatmaps and analysis:

The visualizations clearly indicate that the loops calling list.append consume the most CPU time.

Original English article: https://pythonfiles.wordpress.com/2017/06/01/hunting-performance-in-python-code-part-3/ Translator: buhaoxuesheng