Python Project Engineering: Dependency Management, Project Structure, Modules, Type Checking, Formatting, Configuration, Exception Handling, and Testing

Dependency Management

Before PEP 518 and pyproject.toml , projects could not declare the build tools required by tools like pip. Setuptools introduced a setup_require parameter, but it required setuptools to be installed beforehand, creating a chicken‑egg problem.

PEP 518 allows you to specify the build tool and its version, solving the issue of ambiguous requirements.txt files that cannot differentiate between production, development, and testing dependencies, nor declare Python version or system requirements.

In this internal project we chose PDM , a next‑generation Python package manager. If you already use pipenv or poetry and are satisfied, you can keep them; otherwise, PDM may provide missing features.

Poetry is also a solid choice; it combines virtual‑environment management and packaging, acting as a superset of Pipenv and Flit.

When using PDM or Poetry, create a .venv (or similar) folder via virtualenv . We prefer virtualenv over PEP 582 because many systems rely on a default Python interpreter, and PEP 582 would force the use of the system Python, which is not always desirable, and VS Code does not yet support it.

These tools let you separate dependencies for development, testing, and production environments, and you can add custom commands (e.g., start , test ) to pyproject.toml via PDM scripts, ensuring the package runs from the project root.

Project Structure

A recommended layout is:

Dockerfile – used to build a Docker image for production deployment.

docs – documentation files.

LICENSE – open‑source license file.

pyproject.toml – PEP 518 configuration containing project metadata, dependencies, and build‑tool settings.

README.md – project overview and usage instructions.

{project_name} – the main source code directory (any name that does not clash with third‑party packages).

{project_name}-stubs – optional directory for mypy stub files when the project is a library.

tests – unit tests and related files.

tox.ini – configuration for the tox testing tool.

.gitignore – specifies files Git should ignore.

Module Import

Python modules are the primary abstraction layer. Separate code into logical parts, e.g., a user‑interface layer and a data‑access layer, and import the latter using import or from ... import . Module names should be short, lowercase, and avoid special characters like dots or question marks.

Example of a good import style:

<code># OK
import library.plugin.foo
# not OK
import library.foo_plugin
</code>

When importing a module, Python searches for a matching modu.py in the current directory, then in PYTHONPATH . Once found, the module is executed in an isolated namespace, and its classes, functions, and variables become accessible via that namespace.

Prefer explicit imports such as from modu import func over wildcard imports ( from modu import * ) for readability and maintainability. For intra‑project imports, use relative syntax ( from . import modu ) instead of absolute ( from my import modu ).

Expose public symbols in __init__.py and avoid placing substantial code there; keep it minimal to prevent unnecessary execution when importing nested packages.

Type Checking

Python is dynamically typed, which can lead to unclear function signatures. Since Python 3.5, PEP 484 introduced type hints. mypy is a static type checker that supports inference, generics, callables, tuples, unions, and structural subtyping. Declare parameter and return types, using Optional or the | None syntax for nullable values.

<code>def register(self, factory: Optional[PooledObjectFactory] = None, name: Optional[str] = None) -> None:
    pass
</code>

Install the mypy extension in VS Code for inline type checking.

Code Formatting and Linting

PEP 8 defines the official Python style guide. Tools like pep8 (now pycodestyle ) check compliance. Black is an “uncompromising” formatter that automatically rewrites non‑conforming code, producing minimal diffs and integrating well with modern IDEs.

Configuration Management

Store configuration files in {project_name}/{project_name} using YAML for better compatibility with systems like Kubernetes. Deserialize YAML into a Python dataclass or regular class, avoiding raw dict manipulation for mutable configuration structures.

Exception Management

All languages have exceptions; they help pinpoint errors. Define custom exceptions for domain‑specific error conditions. Use try...except blocks to catch specific exceptions (e.g., ValueError ) rather than the broad Exception base class. Log errors with an ERROR suffix and include meaningful messages.

Testing

Python offers built‑in testing frameworks and many third‑party options. Follow naming conventions: test files start with test_ , test functions start with test_ , and test classes start with Test . Place tests under a tests directory.

pytest extends unittest with powerful syntax sugar and plugins. tox manages multiple isolated virtual environments, enabling testing across different Python versions.

Promotional Section (Skipped in Academic Summary)

The original article includes QR‑code images and links to free Python courses and resources, which are promotional and not part of the technical content.