Master Python Coroutines: From Basics to Real-World File Search

1. Coroutines

1.1 Concept of Coroutines A coroutine (also called a micro‑thread or fiber) is a lightweight user‑level thread. Unlike OS‑level threads, coroutines are scheduled by the program itself. A function can be paused at a yield point and later resumed, preserving its local state.

1.2 Advantages and Disadvantages

Advantages: no OS thread‑switch overhead, no need for atomic locks, easier control‑flow, high concurrency with low cost.

Disadvantages: runs in a single thread so cannot exploit multiple CPU cores, blocking I/O blocks the whole program.

2. Implementing Coroutines in Python

2.1 Using yield to create a coroutine

def eater(name):
    print("%s eat food" % name)
    while True:
        food = yield
    print("done")

g = eater("gangdan")
print(g)  # <generator object eater at ...>

Running the generator shows that g is a generator object.

2.2 Sending values into a coroutine

def creater(name):
    print("%s start to eat food" % name)
    food_list = []
    while True:
        food = yield food_list
        print("%s get %s ,to start eat" % (name, food))
        food_list.append(food)

builder = creater("tom")
next(builder)               # initialise, stop at first yield
print(builder.send("包子"))   # -> ['包子']
print(builder.send("骨头"))   # -> ['包子', '骨头']
print(builder.send("菜汤"))   # -> ['包子', '骨头', '菜汤']

Each call to send() resumes the coroutine, passes a value to the yield expression, and returns the current food_list.

2.3 Decorating a coroutine to auto‑initialise

def init(func):
    def wrapper(*args, **kwargs):
        builder = func(*args, **kwargs)
        next(builder)  # automatically advance to first yield
        return builder
    return wrapper

@init
def creater(name):
    print("%s start to eat food" % name)
    food_list = []
    while True:
        food = yield food_list
        print("%s get %s ,to start eat" % (name, food))
        food_list.append(food)

Now the coroutine is ready to receive values immediately after creation.

2.4 Real‑world application: searching files for a pattern

The following generator pipeline mimics grep -rl using coroutines:

@init
def search():
    while True:
        dir_name = yield
        for root, _, files in os.walk(dir_name):
            for file in files:
                yield f"{root}\{file}"

@init
def opener(target):
    while True:
        path = yield
        with open(path, 'rb') as f:
            target.send((path, f))

@init
def cat(target):
    while True:
        path, f = yield
        for line in f:
            if target.send((path, line)):
                break

@init
def grep(pattern, target):
    while True:
        path, line = yield False
        if pattern in line:
            target.send(path)
            yield True
        else:
            yield False

@init
def printer():
    while True:
        path = yield
        print(path)

# Build pipeline
pipeline = search(opener(cat(grep(b'error', printer())) )
pipeline.send(r'E:\Python\script')

This pipeline walks a directory tree, opens each file, reads lines, filters those containing the byte pattern b'error', and prints the matching file paths.

3. Procedural Programming Perspective

From a procedural viewpoint, the program is a linear sequence of steps: discover file paths, open files, read lines, filter, and output results. This style yields clear architecture but limited extensibility.

Overall, the article demonstrates how coroutines can simplify asynchronous control flow and be combined to build complex data‑processing pipelines.