CPython Drops Mandatory Rust Dependency, Opts for Optional Rust API in Python 3.16

Background: CPython's Rust Integration Proposal

In November 2023, Python core developer Emma Smith proposed integrating Rust into CPython, initially for writing extension modules but with a long-term goal of making Rust a required dependency across the CPython codebase. This mirrored the relationship between Rust and C, as CPython is largely built on C, which has manual memory management risks that Rust's compile-time memory safety could eliminate.

Community Concerns and Revised Proposal

The proposal faced immediate pushback. Developer Liam Provin highlighted several issues: not all Python implementations run on platforms supported by Rust; building Rust itself requires Python, creating a circular dependency; and forced upgrades typically provoke user resistance. In May 2024, Smith revised the proposal, dropping the mandatory dependency requirement and deferring it to a future discussion. The new direction focuses on optional Rust-based extension modules that developers can enable or ignore.

Guido van Rossum's Endorsement

Python creator Guido van Rossum supported the incremental approach, commenting that a full Rust rewrite is impractical but introducing Rust in less critical components first, then gradually expanding its role, is a sound plan.

Current Implementation Plan for Python 3.16

Core developers are now working on integrating Rust into the build process and designing a Rust API to allow Rust crates to build parts of Python. The API is slated for Python 3.16 (expected October 2027) and will include the Rust zlib compression library as a test crate. Rust project manager Tomáš Šedovič, speaking at RustConf in Montreal, explained that this optional approach avoids many developer concerns.

Technical Challenges

GCC support: Rust must support the GNU Compiler Collection for the many niche platforms CPython runs on.

Build system differences: Rust and Python handle standard library and dynamic shared library builds differently, requiring harmonization.

Cross-language sanitizer: A "cleaner" mechanism is needed to manage memory allocation differences between the two languages; projects like BorrowSanitizer are addressing similar issues in C/C++ and Rust integration.

Enhancing Rust's Drop trait: The most complex challenge is extending Rust's Drop trait so Python objects can access runtime context when being deallocated, requiring significant Rust engineering effort.

Python Adopting Rust's Tooling Innovations

The original proposal praised Cargo as an excellent build system that automates the entire development lifecycle—dependency fetching, compilation, linking, and testing—in one tool. Python currently relies on separate tools for project initialization, building, testing, and publishing. Python has already begun emulating Cargo with uv, a unified packaging and project management tool written in Rust. Further adoption of Rust-inspired tooling is anticipated.

Guido van Rossum even joked that CPython might eventually be renamed "CRPython."