Advanced Python Tricks: List Comprehensions, Async IO, Decorators & More

Python is one of the most popular programming languages today; its simple syntax and powerful ecosystem attract many beginners, but developers who use Python efficiently in work and research rely on a series of advanced techniques.

1. Advanced Syntax and Code Style

1.1 List Comprehensions and Generator Expressions

Pythonic style emphasizes brevity. List comprehensions and generator expressions are representative tools.

# List comprehension
squares = [x**2 for x in range(10) if x % 2 == 0]

# Generator expression (memory‑saving)
squares_gen = (x**2 for x in range(10) if x % 2 == 0)

Difference:

List comprehension: Generates the full list at once, high memory usage.

Generator expression: Computes on demand, saves memory, suitable for large data streams.

1.2 Unpacking and the "*" Operator

Python's unpacking syntax makes function argument passing and data structure splitting more flexible.

a, b, *rest = [1, 2, 3, 4, 5]
print(a, b, rest)  # Output: 1 2 [3, 4, 5]

In function parameters:

def func(x, y, z):
    print(x, y, z)

args = [10, 20, 30]
func(*args)  # Equivalent to func(10, 20, 30)

2. Data Structures and Algorithm Tricks

2.1 collections Module

The standard library collections provides many advanced data structures:

deque: Efficient double‑ended queue.

Counter: Counting elements.

defaultdict: Dictionary with default values.

from collections import Counter, defaultdict, deque

# Counter
cnt = Counter("abracadabra")
print(cnt.most_common(2))  # [('a', 5), ('b', 2)]

# defaultdict
dd = defaultdict(int)
dd["key"] += 1

Use cases:

Counter → text statistics

deque → queue / sliding window

defaultdict → auto‑initialized nested dicts

2.2 heapq and Priority Queues

heapq

lets us easily implement min‑heap or max‑heap, suitable for Top‑K problems.

import heapq
nums = [1, 8, 3, -5, 7]
heapq.heapify(nums)  # Transform into a min‑heap
print(heapq.nsmallest(3, nums))  # [-5, 1, 3]

2.3 itertools Combination Tricks

itertools

is a powerful tool for permutations, combinations, and iteration.

import itertools
# Permutations
print(list(itertools.permutations([1, 2, 3], 2)))
# Combinations
print(list(itertools.combinations([1, 2, 3], 2)))

3. Functional Programming Concepts

3.1 lambda and Higher‑Order Functions

nums = [1, 2, 3, 4]
squares = list(map(lambda x: x**2, nums))

3.2 functools Utilities

functools

provides reduce, partial, etc.

from functools import reduce, partial

# reduce
prod = reduce(lambda x, y: x*y, [1, 2, 3, 4])

# partial
def power(base, exp):
    return base ** exp
square = partial(power, exp=2)
print(square(5))  # 25

3.3 Decorators

Decorators allow adding functionality without modifying the original function.

import time

def timer(func):
    def wrapper(*args, **kwargs):
        start = time.time()
        res = func(*args, **kwargs)
        end = time.time()
        print(f"运行时间: {end-start:.4f}s")
        return res
    return wrapper

@timer
def slow_func():
    time.sleep(1)
    return "Done"

slow_func()

4. Parallel and Asynchronous Programming

4.1 Multithreading and Multiprocessing

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

def work(x):
    return x*x

with ThreadPoolExecutor(max_workers=4) as executor:
    results = list(executor.map(work, range(10)))

4.2 Asyncio

Asynchronous I/O is a core capability of Python 3.

import asyncio

async def task(name):
    await asyncio.sleep(1)
    print(f"Task {name} finished")

async def main():
    await asyncio.gather(task(1), task(2), task(3))

asyncio.run(main())

5. Object‑Oriented Design and Patterns

5.1 Data Classes

from dataclasses import dataclass

@dataclass
class Point:
    x: int
    y: int

p = Point(1, 2)
print(p)  # Point(x=1, y=2)

5.2 Singleton Pattern

class Singleton:
    _instance = None
    def __new__(cls, *args, **kwargs):
        if not cls._instance:
            cls._instance = super().__new__(cls)
        return cls._instance

6. Performance Optimization Techniques

6.1 Built‑in Functions and NumPy

Python loops are slow; use vectorized operations.

import numpy as np
arr = np.arange(1e6)
print(arr.sum())  # Fast C implementation

6.2 Memory Profiling

Use sys.getsizeof to estimate object size.

import sys
print(sys.getsizeof([1, 2, 3]))  # 88 bytes

6.3 Cython and Numba

JIT compilation can accelerate numeric computations.

from numba import jit

@jit(nopython=True)
def f(x):
    return x**2 + x + 1

print(f(10))

7. Debugging and Visualization

7.1 logging

Compared with print, logging is more suitable for production environments.

import logging
logging.basicConfig(level=logging.INFO)
logging.info("程序启动")

7.2 Visual Debugging

Using matplotlib or graphviz to visually analyze data structures and algorithm processes.

import matplotlib.pyplot as plt
x = np.linspace(0, 10, 100)
y = np.sin(x)
plt.plot(x, y)
plt.show()

8. Comprehensive Example: Asynchronous Web Crawler

Combining the above techniques, a simple asynchronous crawler is implemented.

import asyncio
import aiohttp

urls = ["https://example.com"] * 5

async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            return await resp.text()

async def main():
    tasks = [fetch(url) for url in urls]
    results = await asyncio.gather(*tasks)
    print(f"抓取完成，共 {len(results)} 页")

asyncio.run(main())

The example demonstrates async programming, context managers, and gather for concurrent scheduling, showcasing Python's advanced capabilities.

The article ties together many advanced Python usages, helping readers write cleaner, more efficient code and significantly improve productivity in engineering practice.