How Python’s ‘with’ Statement Revolutionized Resource Management

Abstract

PEP 343 proposes adding a new with statement to Python to replace the conventional try/finally pattern for resource management.

Author's Notes

The original draft was written by Guido van Rossum; Nick Coghlan later updated it based on discussions on python‑dev.

Introduction

After extensive debate over PEP‑340, the author withdrew that proposal and introduced a variant that uses the with keyword. The final design adds a context‑manager protocol consisting of __enter__() and __exit__() methods.

Motivation and Summary

PEP‑340 introduced powerful ideas such as using generators as block templates, but its loop‑like structure caused controversy because it allowed break and continue inside the block. The with statement provides a clearer, non‑loop control flow while still allowing acquisition and release of resources.

Specification: ‘with’ Statement

The syntax is:

with EXPR as VAR:
    BLOCK

Here EXPR must evaluate to a context‑manager object that implements __enter__ and __exit__. The as VAR clause is optional.

Example Implementations

Lock template:

@contextmanager
def locked(lock):
    lock.acquire()
    try:
        yield
    finally:
        lock.release()

File opening template:

@contextmanager
def opened(filename, mode="r"):
    f = open(filename, mode)
    try:
        yield f
    finally:
        f.close()

Database transaction template:

@contextmanager
def transaction(db):
    db.begin()
    try:
        yield None
    except:
        db.rollback()
        raise
    else:
        db.commit()

Generator‑based decorator (simplified):

class GeneratorContextManager(object):
    def __init__(self, gen):
        self.gen = gen
    def __enter__(self):
        try:
            return self.gen.next()
        except StopIteration:
            raise RuntimeError("generator didn't yield")
    def __exit__(self, typ, value, tb):
        if typ is None:
            try:
                self.gen.next()
            except StopIteration:
                return True
        else:
            try:
                self.gen.throw(typ, value, tb)
                raise RuntimeError("generator didn't stop after throw()")
            except StopIteration:
                return True

Standard library context managers (e.g., file, threading.Lock, decimal contexts) use the same protocol.

Transition Plan

In Python 2.5 the new syntax required from __future__ import with_statement. Starting with Python 2.6 the keywords with and as are always available.

References

PEP 343 – The "with" Statement (https://www.python.org/dev/peps/pep-0343)

PEP 342 – Enhancing Generators (generator‑based context managers)