Comprehensive Guide to Python Version, Environment, Package Management and Build Tools

Background Knowledge

When I first started using Python and created my first package, I was confused. Managing packages and environments turned out to be more difficult than expected, and many tools exist without a clear guide on which to choose. This article aims to give a complete overview and structured evaluation of these tools.

Tool Classification

The tools are divided into five important categories for environment and package management:

Python version management

Package management

Environment management (mainly virtual environments)

Package building

Package publishing

As shown in the Venn diagram below, many tools serve single purposes while others are multi‑purpose.

Python Version Management

Definition

A tool for Python version management lets you easily install different Python versions and switch between them.

Motivation

Multiple projects may require different Python versions, or you may want to test a library against several versions, explore new features, or test pre‑release builds.

Tools

The Venn diagram shows four tools for this category: pyenv , conda , rye and pyflow . We will first look at pyenv as a single‑purpose tool.

<code># Install a specific Python version
pyenv install 3.10.4

# Switch Python version for the current shell
pyenv shell <version>

# Use a version automatically in a directory
pyenv local <version>

# Set a global Python version for the user
pyenv global <version>
</code>

(Virtual) Environment Management

Definition

Environment management tools allow you to create and manage (virtual) environments.

Motivation

Projects often have conflicting dependencies, so isolating them in separate environments avoids clashes. Installing packages system‑wide can cause permission issues; using --user helps but is not always known to beginners.

Tools

Common tools include venv , virtualenv , pipenv , conda , pdm , poetry , hatch , rye and pyflow . Only venv and virtualenv are single‑purpose.

venv

The built‑in venv module creates virtual environments. Important commands:

<code># Create a new environment
python3 -m venv <env_name>

# Activate the environment
. <env_name>/bin/activate

# Deactivate the environment
deactivate
</code>

virtualenv

virtualenv improves on venv with more features and speed. Commands are similar:

<code># Create a new environment
virtualenv <env_name>

# Activate the environment
. <env_name>/bin/activate

# Deactivate the environment
deactivate
</code>

Package Management

pyproject.toml

The most important file for packaging is pyproject.toml . It defines project metadata, build system requirements, and dependencies. Below is a simplified example from pandas:

<code>[build-system]
requires = ["meson-python==0.13.1", "meson==1.2.1", "wheel", "Cython==3.0.5"]
build-backend = "mesonpy"

[project]
name = "pandas"
dynamic = ["version"]
description = "Powerful data structures for data analysis, time series, and statistics"
readme = "README.md"
authors = [{name = "The Pandas Development Team", email = "[email protected]"}]
license = {file = "LICENSE"}
requires-python = ">=3.9"
dependencies = [
    "numpy>=1.22.4,<2.0.0.dev0",
    "python-dateutil>=2.8.2",
    "pytz>=2020.1",
    "tzdata>=2022.7"
]
</code>

.lock Files

Lock files (e.g., poetry.lock ) record exact versions of all installed dependencies, ensuring reproducible builds across platforms.

<code># Example lock entry generated by Poetry
[[package]]
name = "build"
version = "1.0.3"
description = "A simple, correct Python build frontend"
optional = false
python-versions = ">=3.7"
files = [{file = "build-1.0.3-py3-none-any.whl", hash = "sha256:..."}]
</code>

Motivation

Packages let you structure code using the package.module syntax and share it with others. The pyproject.toml file declares dependencies, so collaborators can install the package without manually handling each requirement.

Tools

Popular package managers include pip , pipx , pipenv , conda , pdm , poetry , rye and pyflow . The most widely known is pip :

<code>pip install <package_name>
</code>

Multi‑purpose Tools

pipenv

pipenv combines pip and virtualenv . It introduces two additional files: Pipfile (a TOML file defining dependencies) and Pipfile.lock (a deterministic lock file).

<code># Install a package
pipenv install <package_name>

# Run a script inside the virtual environment
pipenv run <script_name.py>

# Spawn a shell in the virtual environment
pipenv shell
</code>

Conda

Conda is a general‑purpose package manager that handles both Python and non‑Python packages. It also supports building and publishing conda packages, which differ from PyPI wheels.

Feature Evaluation

We compare tools across several dimensions:

Does the tool manage dependencies?

Does it resolve/lock dependencies?

Does it provide a clean build/publish workflow?

Does it support plugins?

Does it support PEP 660 (editable installs)?

Does it support PEP 621 (project metadata)?

Flit

Flit focuses on building and publishing pure‑Python packages. It does not manage dependencies or environments.

Dependency management: ❌

Package management: ❌

Environment management: ❌

Build & publish: ✅

Plugins: ❌

PEP 660: ✅

PEP 621: ✅

<code># Create a new pyproject.toml
flit init

# Build and publish
flit publish
</code>

Poetry

Poetry handles everything except Python version management. It manages dependencies, resolves/locks them, provides clean build/publish, supports plugins, and implements PEP 660, but lacks PEP 621 support.

Dependency management: ✅

Dependency lock: ✅

Build & publish: ✅

Plugins: ✅

PEP 660: ✅

PEP 621: ❌

<code># Create a new project structure and pyproject.toml
poetry new <project_name>

# Interactive pyproject.toml creation
poetry init

# Install dependencies
poetry install

# Add a dependency
poetry add <package_name>

# Show dependency tree
poetry show --tree

# Activate virtual env
poetry shell

# Run a script inside the env
poetry run python <script_name.py>

# Build package
poetry build

# Publish to PyPI
poetry publish

# Update dependencies
poetry update
</code>

pdm

pdm is a newer tool inspired by Poetry and PyFlow. It supports dependency management, locking, clean build/publish, plugins, and both PEP 660 and PEP 621. It also implements PEP 582 (local packages) as an optional environment management method.

Dependency management: ✅

Dependency lock: ✅

Build & publish: ✅

Plugins: ✅

PEP 660: ✅

PEP 621: ✅

<code># Interactive pyproject.toml creation
pdm init

# Install packages
pdm install

# Add a dependency (no immediate install)
pdm add <package_name>

# Show dependency graph
pdm list --graph

# Run a script in the current env
pdm run python <script_name.py>

# Update dependencies
pdm update

# Build and publish
pdm build
pdm publish
</code>

Hatch

Hatch focuses on environment management and building/publishing. It does not manage dependencies or lock files yet (planned). It supports plugins, PEP 660 and PEP 621.

Dependency management: ❌

Dependency lock: ❌

Build & publish: ✅

Plugins: ✅

PEP 660: ✅

PEP 621: ✅

<code># Create a new project and pyproject.toml
hatch new <project_name>

# Initialize an existing project
hatch new --init

# Show dependencies (must be added manually to pyproject.toml)
hatch dep show table

# Activate virtual env
hatch shell

# Run a script inside the env
hatch run python <script_name.py>

# Build and publish
hatch build
hatch publish
</code>

Rye

Rye, written in Rust, covers all tasks: version management, package management, environment management, building and publishing. It currently lacks a plugin interface.

Dependency management: ✅

Dependency lock: ✅

Build & publish: ✅

Plugins: ❌

PEP 660: ✅

PEP 621: ✅

<code># Create a new project and pyproject.toml
rye init <project_name>

# Pin a specific Python version
rye pin 3.10

# Add a dependency (does not install immediately)
rye add <package_name>

# Sync environment and lock file, installing dependencies
rye sync

# Activate virtual env
rye shell

# Run a script inside the env
rye run python <script_name.py>

# Build and publish
rye build
rye publish
</code>

Summary

Flit

Poetry

pdm

Hatch

rye

Manages dependencies

❌

✅

✅

❌

✅

Resolves/locks dependencies

❌

✅

✅

❌

✅

Clean build/publish workflow

✅

✅

✅

✅

✅

Supports plugins

❌

✅

✅

✅

❌

Supports PEP 660

✅

✅

✅

✅

✅

Supports PEP 621

✅

❌

✅

✅

✅