Master Python Iterators and Generators: From Basics to Advanced Usage

Iterators

Introduction

Iterator is a tool for iterating values; iteration is a repeated feedback process.

while True:
    msg = input('>>: ').strip()
    print(msg)

num_list = [0, 1, 2, 3, 4, 5]
count = 0
while count < len(num_list):
    print(num_list[count])
    count += 1

Iterable Objects

What is an iterable object

Objects with a built‑in __iter__ method are iterable.

Eight basic data types

# 1. Numeric types
# 1.1 int — not iterable
num = 1
print(num.__iter__)  # AttributeError

# 1.2 float — not iterable
num_float = 1.0
print(num_float.__iter__())  # AttributeError

# 2. str — iterable
name = 'chosen'
print(name.__iter__)      # <method-wrapper '__iter__' of str object ...>
print(name.__iter__())    # <str_iterator object ...>

# 3. bool — not iterable
is_right = False
print(is_right.__iter__)  # AttributeError

# 4. list — iterable
name_list = [1, 2, 3]
print(name_list.__iter__)      # <method-wrapper '__iter__' of list object ...>
print(name_list.__iter__())   # <list_iterator object ...>

# 5. dict — iterable
info_dict = {"name": "chosen"}
print(info_dict.__iter__)      # <method-wrapper '__iter__' of dict object ...>
print(info_dict.__iter__())   # <dict_keyiterator object ...>

# 6. tuple — iterable
num_tuple = (1,)
print(num_tuple.__iter__)      # <method-wrapper '__iter__' of tuple object ...>
print(num_tuple.__iter__())   # <tuple_iterator object ...>

# 7. set — iterable
num_set = {1}
print(num_set.__iter__)      # <method-wrapper '__iter__' of set object ...>
print(num_set.__iter__())   # <set_iterator object ...>

Summary

Non‑iterable types: int, float, bool

Iterable types: str, list, dict, tuple, set

Iterator Objects

What is an iterator object

Calling obj.__iter__() returns an iterator object.

Iterator objects have iter and next methods; files are iterator objects.

Iterators provide a unified, index‑free way to retrieve values from any collection.

Eight basic data types (iterator examples)

# str
name_str = 'chosen'
name_iter = name_str.__iter__()
name_iter_two = iter(name_str)
print(name_iter)
print(name_iter_two)
print(name_iter.__next__)  # c
print(next(name_iter))    # h

# list
name_list = [1, 2, 3]
name_list_iter = iter(name_list)
print(name_list_iter)
print(name_list_iter.__next__)  # 1
print(next(name_list_iter))    # 2

# dict
info_dict = {"name": "chosen", "age": 18}
info_dict_iter = iter(info_dict)
print(info_dict_iter)
print(info_dict_iter.__next__)  # name
print(next(info_dict_iter))    # age

# tuple
num_tuple = (1, 2)
num_tuple_iter = iter(num_tuple)
print(num_tuple_iter)
print(num_tuple_iter.__next__)  # 1
print(next(num_tuple_iter))    # 2

# set
num_set = {1, 2, 3}
num_set_iter = iter(num_set)
print(num_set_iter)
print(num_set_iter.__next__)  # 1
print(next(num_set_iter))    # 2

Summary

# Objects with __iter__() and __next__() are iterator objects.
# In the eight basic types, everything except int, float, bool is an iterator.
# An iterator object is always iterable.
# An iterable object has __iter__(); an iterator object has both __iter__ and __next__.

Pros and Cons of Iterators

Advantages

Provides a unified way to iterate over sequence and non‑sequence types.

No need to use indexes.

State is saved after each value, so the next call continues from the current position.

Disadvantages

Length cannot be obtained unless the iterator is exhausted.

Can only move forward; cannot restart without creating a new iterator.

Re‑using the same iterator in multiple loops will cause only one loop to receive values.

Generators

Generators produce elements one by one, avoiding the need to build the whole sequence in memory.

They are memory‑efficient for large data sets.

# Suppose a database has 100 million rows.
# Using read() would exhaust memory.
# Using a generator to fetch 100 rows at a time saves memory.

Creating Generators

List comprehension

Replace brackets with parentheses to create a generator.

# List comprehension
start_list = [x * 2 for x in range(5)]
print(start_list)  # [0, 2, 4, 6, 8]

# Generator expression
G = (x * 2 for x in range(5))
print(G)  # <generator object <genexpr> ...>
print(list(G))  # [0, 2, 4, 6, 8]

yield keyword

Defining a generator function with yield pauses execution and returns a value; subsequent calls resume execution.

def my_generator():
    yield 1
    yield 2
    yield 3

g = my_generator()
print(next(g))  # 1
print(next(g))  # 2
print(next(g))  # 3

Generator Cases

def eater():
    print('Start eating ovo')
    while True:
        food = yield
        print(f'Got food: {food}, start eating: {food}')

res = eater()
print(res)  # <generator object eater ...>
print(res.__next__())  # prints start message, returns None
print(res.__next__())  # prints food None
res.send("Fish")
print(res.__next__())  # None
print(res.send("Kung Pao Chicken"))  # prints food

Calling function

res = eater()
print(res)
print(res.__next__())
print(res.__next__())

Sending values to generator

res = eater()
print(res)
print(res.__next__())
res.send("Fish")
print(res.__next__())
print(res.send("Kung Pao Chicken"))

Decorator + Generator

def init_iter(func):
    def inner(*args, **kwargs):
        g = func(*args, **kwargs)
        next(g)
        return g
    return inner

@init_iter
def eater():
    print('Start eating ovo')
    while True:
        food = yield
        print(f'Got food: {food}, start eating: {food}')

res = eater()
res.send("Fish")
res.send("Kung Pao Chicken")

Generator modifying mutable data

def init_iter(func):
    def inner(*args, **kwargs):
        g = func(*args, **kwargs)
        next(g)
        return g
    return inner

@init_iter
def eater():
    print('Start eating ovo')
    food_list = []
    while True:
        food = yield
        food_list.append(food)
        print(f'Got food: {food}, start eating: {food}')
        print(f'Current kitchen: {food_list}')

res = eater()
res.send("Fish")
res.send("Kung Pao Chicken")
# Output shows food_list accumulating items.

yield + next usage

def my_range(start, stop, step):
    print("start")
    while start < stop:
        yield start
        start += step
    print("end")

res = my_range(0, 5, 1)
print(res)
print(res.__next__())  # 0
print(res.__next__())  # 1
print(res.__next__())  # 2
print(res.__next__())  # 3
print(res.__next__())  # 4
print(res.__next__())  # StopIteration