Replacing Python Source Map Processing with Rust at Sentry: Performance Gains and Integration Details

Sentry, a cloud service for error monitoring and analysis handling over a billion errors monthly, found its Python‑based source‑map processing pipeline becoming a major performance bottleneck as source maps grew larger and more complex.

The investigation revealed that deserializing millions of source‑map tokens into Python objects caused massive memory consumption (up to ~800 MB for a 30 MB map), millions of allocations, and heavy garbage‑collection overhead, leaving little room for optimization within Python itself.

To address this, the team rewrote the parser in Rust, creating a thin wrapper library called libsourcemap . Benchmarks showed processing time dropping from >20 seconds to <0.5 seconds, with CPU utilization and memory usage dramatically reduced.

After deploying the Rust library, the worst‑case processing time fell to one‑tenth of the original, average processing time settled around 400 ms, and overall end‑to‑end event handling time for JavaScript stacks decreased to roughly 300 ms.

Integration was achieved by compiling the Rust code into a dynamic library (dylib) and exposing a C ABI using the #[no_mangle] attribute and panic handling helpers. The library is then loaded from Python via CFFI, with a setup.py script that builds the Cargo project, preprocesses C headers, generates shim code, and loads the module at runtime.

The Python side provides a context manager and custom error classes to translate Rust panics into Python exceptions, and defines a base class for source‑map handling that delegates to the Rust implementation.

Key successes include the seamless combination of Rust, Python, and CFFI, reproducible builds using Docker on older CentOS versions, and the robust Rust ecosystem (serde, base64, memmap). Challenges involved long compilation times for each change and fragile setuptools steps that required careful handling.

Future work aims to further optimise by caching parsed structures in memory, leveraging mmap for file‑system caching, and exploring Rust for other CPU‑intensive workloads, while recognising that many remaining tasks are I/O bound.

In conclusion, the project demonstrated that Rust is an excellent tool for high‑performance native extensions, delivering substantial speedups and lower resource usage with minimal implementation effort, while Python remains the primary language for most of Sentry's codebase.