10 Essential Python Design Patterns Every Developer Should Master
Explore ten crucial Python design patterns—from Singleton to Memoization—complete with real-world examples for data pipelines, ETL processes, and scalable systems, helping developers write cleaner, more maintainable, and efficient code across backend and data engineering tasks.
Python is loved for its simplicity and readability, but writing clean, maintainable, and scalable code often depends on understanding key coding patterns—reusable solutions to common programming problems.
Here we explore ten important Python coding patterns with practical examples valuable for data scientists, backend developers, and automation engineers.
1. Singleton Pattern
What it is:
Ensures a class has only one instance and provides a global access point.
Practical example:
Managing database connections in a data pipeline—multiple connections can overload the server.
<code>class SingletonDB:
_instance = None
def __new__(cls, conn_str):
if not cls._instance:
cls._instance = super().__new__(cls)
cls._instance.conn_str = conn_str
return cls._instance
</code>Use case: Avoid reconnecting to the database multiple times in the same script.
2. Factory Pattern
What it is:
Creates objects without specifying the exact class, ideal for handling conditional logic.
Practical example:
Connecting to different databases based on environment (e.g., SQLite for development, PostgreSQL for production).
<code>class DatabaseFactory:
def get_connection(db_type):
if db_type == 'sqlite':
import sqlite3
return sqlite3.connect('my.db')
elif db_type == 'postgres':
import psycopg2
return psycopg2.connect("dbname=test user=postgres")
</code>Use case: Configurable, extensible ETL pipelines.
3. Builder Pattern
What it is:
Separates the construction of a complex object from its representation.
Practical example:
Building dynamic SQL queries in a clear and reusable way.
<code>class SQLBuilder:
def __init__(self):
self.query = "SELECT * FROM users"
def where(self, condition):
self.query += f" WHERE {condition}"
return self
def order_by(self, column):
self.query += f" ORDER BY {column}"
return self
sql = SQLBuilder().where("age > 25").order_by("name").query
</code>Use case: Automated dashboard queries based on user‑defined filters.
4. Decorator Pattern
What it is:
Adds new functionality to a function by wrapping it, without modifying the original function.
Practical example:
Logging each SQL query execution in an analytics application.
<code>def log_query(func):
def wrapper(*args, **kwargs):
print(f"Running query: {func.__name__}")
return func(*args, **kwargs)
return wrapper
@log_query
def get_data():
print("SELECT * FROM users")
</code>Use case: Adding audit logs or performance tracing.
5. Iterator Pattern
What it is:
Provides a way to access elements sequentially without exposing the underlying structure.
Practical example:
Paginating through large SQL result sets.
<code>def fetch_in_batches(cursor, batch_size=100):
while True:
results = cursor.fetchmany(batch_size)
if not results:
break
yield from results
</code>Use case: Streaming millions of rows from a database without loading everything into memory.
6. Strategy Pattern
What it is:
Allows selecting an algorithm at runtime.
Practical example:
Choosing different data‑cleaning methods.
<code>def clean_text_upper(text):
return text.upper()
def clean_text_lower(text):
return text.lower()
def clean_text(text, strategy):
return strategy(text)
clean_text("Hello", clean_text_lower) # returns 'hello'
</code>Use case: Modular preprocessing in NLP or machine‑learning pipelines.
7. Observer Pattern
What it is:
Notifies all dependent objects when the state of an object changes.
Practical example:
Automatically updating a dashboard when the data source changes.
<code>class Observable:
def __init__(self):
self.subscribers = []
def subscribe(self, func):
self.subscribers.append(func)
def notify(self, data):
for sub in self.subscribers:
sub(data)
def on_data_change(data):
print(f"Dashboard updated with: {data}")
obs = Observable()
obs.subscribe(on_data_change)
obs.notify("new_data.csv")
</code>Use case: Real‑time analytics dashboards or data monitoring.
8. Command Pattern
What it is:
Encapsulates a request as an object, allowing execution, undo, logging, or scheduling.
Practical example:
Batch‑executing ETL commands.
<code>class LoadData:
def execute(self):
print("Loading data...")
class CleanData:
def execute(self):
print("Cleaning data...")
pipeline = [LoadData(), CleanData()]
for step in pipeline:
step.execute()
</code>Use case: Modular ETL pipeline design.
9. Template Method Pattern
What it is:
Defines the skeleton of an algorithm, allowing subclasses to override specific steps.
Practical example:
Defining a data‑processing template and customizing steps.
<code>class DataPipeline:
def run(self):
self.extract()
self.transform()
self.load()
def extract(self):
pass
def transform(self):
pass
def load(self):
pass
class SalesPipeline(DataPipeline):
def extract(self):
print("Extracting sales data...")
def transform(self):
print("Cleaning sales data...")
def load(self):
print("Loading into warehouse...")
</code>Use case: Reusable pipeline architecture.
10. Memoization (Caching)
What it is:
Stores results of expensive function calls and reuses them when the same inputs occur again.
Practical example:
Avoiding repeated execution of complex SQL queries.
<code>from functools import lru_cache
@lru_cache(maxsize=128)
def get_customer_data(customer_id):
# Simulate database query
print(f"Querying DB for {customer_id}")
return {"id": customer_id, "name": "Alice"}
</code>Use case: Optimizing performance in dashboards and APIs.
Conclusion
These ten Python coding patterns are powerful tools for any developer’s toolbox. Whether you are building data pipelines, automating dashboards, or developing scalable systems, knowing when and how to apply them will make you a thoughtful and efficient programmer.
Code Mala Tang
Read source code together, write articles together, and enjoy spicy hot pot together.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.