Understanding Iteration, Iterables, Iterators, Generators, Coroutines, Greenlet and Gevent in Python

1. Iteration

1.1 Concept of Iteration

Using a for loop to traverse values is called iteration, e.g., iterating over a list.

<code>for value in [2, 3, 4]:
    print(value)</code>

1.2 Iterable Objects

Standard definition: a class that defines the __iter__ method produces iterable objects such as list, tuple, dict, set, range, and string. Simple memory aid: objects that can be traversed by a for loop are iterable.

1.3 Checking if an Object Is Iterable

<code>from collections import Iterable
result = isinstance((3, 5), Iterable)
print("Tuple is iterable:", result)
result = isinstance([3, 5], Iterable)
print("List is iterable:", result)
result = isinstance({"name": "张三"}, Iterable)
print("Dict is iterable:", result)
result = isinstance("hello", Iterable)
print("String is iterable:", result)
result = isinstance({3, 5}, Iterable)
print("Set is iterable:", result)
result = isinstance(range(5), Iterable)
print("Range is iterable:", result)
result = isinstance(5, Iterable)
print("Integer is iterable:", result)
result = isinstance(5, int)
print("Integer is int type:", result)
class Student(object):
    pass
stu = Student()
result = isinstance(stu, Iterable)
print("Student instance is iterable:", result)
result = isinstance(stu, Student)
print("Student instance is Student type:", result)</code>

1.4 Custom Iterable Objects

Implement the __iter__ method in a class.

Custom Iterable Type Code

<code>from collections import Iterable
class MyList(object):
    def __init__(self):
        self.my_list = list()
    def append_item(self, item):
        self.my_list.append(item)
    def __iter__(self):
        pass
my_list = MyList()
my_list.append_item(1)
my_list.append_item(2)
result = isinstance(my_list, Iterable)
print(result)
for value in my_list:
    print(value)</code>

Execution result shows a TypeError because __iter__ returned None . An iterator is required to iterate over an iterable.

Summary

Providing __iter__ in a class makes its instances iterable; iteration requires an iterator.

2. Iterator

2.1 Custom Iterator Object

A custom iterator defines both __iter__ and __next__ methods.

<code>from collections import Iterable, Iterator
class MyList(object):
    def __init__(self):
        self.my_list = list()
    def append_item(self, item):
        self.my_list.append(item)
    def __iter__(self):
        my_iterator = MyIterator(self.my_list)
        return my_iterator
class MyIterator(object):
    def __init__(self, my_list):
        self.my_list = my_list
        self.current_index = 0
        result = isinstance(self, Iterator)
        print("MyIterator is iterator:", result)
    def __iter__(self):
        return self
    def __next__(self):
        if self.current_index < len(self.my_list):
            self.current_index += 1
            return self.my_list[self.current_index - 1]
        else:
            raise StopIteration
my_list = MyList()
my_list.append_item(1)
my_list.append_item(2)
result = isinstance(my_list, Iterable)
print(result)
for value in my_list:
    print(value)</code>

Result shows the iterator works and prints 1 and 2.

2.2 iter() and next() Functions

iter() obtains an iterator by calling __iter__ ; next() retrieves the next value by calling __next__ .

<code>class MyList(object):
    def __init__(self):
        self.my_list = list()
    def append_item(self, item):
        self.my_list.append(item)
    def __iter__(self):
        my_iterator = MyIterator(self.my_list)
        return my_iterator
class MyIterator(object):
    def __init__(self, my_list):
        self.my_list = my_list
        self.current_index = 0
    def __iter__(self):
        return self
    def __next__(self):
        if self.current_index < len(self.my_list):
            self.current_index += 1
            return self.my_list[self.current_index - 1]
        else:
            raise StopIteration
my_list = MyList()
my_list.append_item(1)
my_list.append_item(2)
my_iterator = iter(my_list)
print(my_iterator)
while True:
    try:
        value = next(my_iterator)
        print(value)
    except StopIteration as e:
        break</code>

2.3 Essence of a for Loop

Iterating Over an Iterable

The for item in iterable loop first calls iter() to get an iterator, then repeatedly calls next() until StopIteration is raised.

Iterating Over an Iterator

When the target is already an iterator, the loop directly calls next() until StopIteration .

2.4 Iterator Application Scenarios

Iterators can generate values on the fly, saving memory. Example: generating Fibonacci numbers.

<code>class Fibonacci(object):
    def __init__(self, num):
        self.num = num
        self.a = 0
        self.b = 1
        self.current_index = 0
    def __iter__(self):
        return self
    def __next__(self):
        if self.current_index < self.num:
            result = self.a
            self.a, self.b = self.b, self.a + self.b
            self.current_index += 1
            return result
        else:
            raise StopIteration
fib = Fibonacci(5)
for value in fib:
    print(value)</code>

Output: 0 1 1 2 3

Summary

An iterator records the current position to retrieve the next value.

3. Generator

3.1 Concept of Generator

A generator is a special iterator that does not require explicit __iter__ and __next__ definitions; it uses yield and works with next() and for loops.

3.2 Creating Generators – Method 1

Replace list comprehension brackets [ ] with parentheses ( ) to create a generator expression.

<code>my_list = [i * 2 for i in range(5)]
print(my_list)
my_generator = (i * 2 for i in range(5))
print(my_generator)
for value in my_generator:
    print(value)</code>

3.3 Creating Generators – Method 2

Using yield inside a function turns it into a generator.

<code>def fibonacci(num):
    a = 0
    b = 1
    current_index = 0
    while current_index < num:
        result = a
        a, b = b, a + b
        current_index += 1
        yield result
fib = fibonacci(5)
value = next(fib)
print(value)
value = next(fib)
print(value)
value = next(fib)
print(value)</code>

3.4 Using return in a Generator

<code>def fibonacci(num):
    a = 0
    b = 1
    current_index = 0
    while current_index < num:
        result = a
        a, b = b, a + b
        current_index += 1
        yield result
        return "haha"
fib = fibonacci(5)
value = next(fib)
print(value)
try:
    value = next(fib)
    print(value)
except StopIteration as e:
    print(e.value)</code>

When return is executed, the generator stops and raises StopIteration with the return value.

3.5 Yield vs Return

yield pauses the function and returns a value; the function can resume later and yield multiple values. return ends the function and returns a single value, causing StopIteration .

3.6 Using send() to Pass Values

send() resumes a generator and can pass a value to the paused yield expression. The first call must use next() or send(None) .

<code>def gen():
    i = 0
    while i < 5:
        temp = yield i
        print(temp)
        i += 1
f = gen()
print(next(f))
print(f.send('haha'))
print(next(f))
print(f.send('haha'))</code>

Summary

Generators are usually created with yield .

yield pauses execution and returns a value; subsequent calls resume where left off.

4. Coroutine

4.1 Concept of Coroutine

A coroutine (micro‑thread) enables multitasking in a single OS thread; a function containing a yield is a coroutine.

Coroutine Implementation with yield

<code>import time

def work1():
    while True:
        print("----work1---")
        yield
        time.sleep(0.5)

def work2():
    while True:
        print("----work2---")
        yield
        time.sleep(0.5)

def main():
    w1 = work1()
    w2 = work2()
    while True:
        next(w1)
        next(w2)
if __name__ == "__main__":
    main()</code>

Output shows alternating execution of work1 and work2.

Summary

Coroutines execute tasks in an alternating order within a single thread.

5. greenlet

5.1 Introduction to greenlet

greenlet wraps coroutine functionality, making task switching easier.

<code>pip3 install greenlet</code>

<code>import time
import greenlet

def work1():
    for i in range(5):
        print("work1...")
        time.sleep(0.2)
        g2.switch()

def work2():
    for i in range(5):
        print("work2...")
        time.sleep(0.2)
        g1.switch()
if __name__ == "__main__":
    g1 = greenlet.greenlet(work1)
    g2 = greenlet.greenlet(work2)
    g1.switch()
</code>

Result shows interleaved execution of work1 and work2.

6. gevent

6.1 Introduction to gevent

gevent builds on greenlet and automatically switches tasks when I/O occurs.

<code>pip3 install gevent</code>

6.2 Basic Usage

<code>import gevent

def work(n):
    for i in range(n):
        print(gevent.getcurrent(), i)

g1 = gevent.spawn(work, 5)
g2 = gevent.spawn(work, 5)
g3 = gevent.spawn(work, 5)
g1.join()
g2.join()
g3.join()
</code>

6.3 Automatic Switching with gevent.sleep

<code>import gevent

def work(n):
    for i in range(n):
        print(gevent.getcurrent(), i)
        gevent.sleep(1)

g1 = gevent.spawn(work, 5)
g2 = gevent.spawn(work, 5)
g3 = gevent.spawn(work, 5)
g1.join()
g2.join()
g3.join()
</code>

6.4 Patching for I/O

<code>import gevent
import time
from gevent import monkey
monkey.patch_all()

def work1(num):
    for i in range(num):
        print("work1....")
        time.sleep(0.2)

def work2(num):
    for i in range(num):
        print("work2....")
        time.sleep(0.2)
if __name__ == '__main__':
    g1 = gevent.spawn(work1, 3)
    g2 = gevent.spawn(work2, 3)
    g1.join()
    g2.join()
</code>

6.5 Using send and joinall

<code>import gevent

def work1():
    for i in range(5):
        print(f"work1工作了{i}")
        gevent.sleep(1)

def work2():
    for i in range(5):
        print(f"work2工作了{i}")
        gevent.sleep(1)
if __name__ == '__main__':
    w1 = gevent.spawn(work1)
    w2 = gevent.spawn(work2)
    gevent.joinall([w1, w2])
</code>

7. Process, Thread, Coroutine Comparison

7.1 Relationship

A process contains at least one thread; a process can have multiple threads.

A thread can host multiple coroutines.

7.2 Comparison

Process: unit of resource allocation.

Thread: unit of OS scheduling.

Process switching is heavy and slow.

Thread switching is moderate (ignoring GIL).

Coroutine switching is lightweight and fast.

Multiple processes/threads can run in parallel on multiple CPUs; coroutines run concurrently within a single thread.

Summary

1. Processes, threads, and coroutines all achieve multitasking; choose based on needs.

2. Threads and coroutines consume fewer resources, so they are used most often.

3. Coroutines require the least resources to create.

Author: y大壮 Source: https://juejin.cn/post/6970520987011907615 For academic sharing only; contact for removal if infringing.

Click Read Original to learn more.