Common Python Pitfalls, Useful Tools, and Performance Tips

1. Commonly Confused Operations

This section compares several Python constructs that are often mixed up.

1.1 Random sampling with and without replacement

import random
random.choices(seq, k=1)  # list of length k, sampling with replacement
random.sample(seq, k)     # list of length k, sampling without replacement

1.2 Lambda function arguments

func = lambda y: x + y          # x is bound at call time
func = lambda y, x=x: x + y     # x is bound at definition time

1.3 copy vs deepcopy

import copy
y = copy.copy(x)      # shallow copy, top‑level only
y = copy.deepcopy(x)  # deep copy, copies all nested objects

When copies and variable aliases are combined, confusion can arise:

a = [1, 2, [3, 4]]

# Alias.
b_alias = a
assert b_alias == a and b_alias is a

# Shallow copy.
b_shallow_copy = a[:]
assert b_shallow_copy == a and b_shallow_copy is not a and b_shallow_copy[2] is a[2]

# Deep copy.
import copy
b_deep_copy = copy.deepcopy(a)
assert b_deep_copy == a and b_deep_copy is not a and b_deep_copy[2] is not a[2]

Modifying an alias affects the original variable; elements in a shallow copy are still references to the original objects, while a deep copy creates independent nested copies.

1.4 Equality (==) vs identity (is)

x == y  # compare values
x is y  # compare object identity

1.5 Type checking

type(a) == int      # ignores inheritance
isinstance(a, int)  # respects inheritance and polymorphism

1.6 String searching

str.find(sub, start=None, end=None)   # returns -1 if not found
str.index(sub, start=None, end=None)   # raises ValueError if not found

1.7 List reverse indexing

Python allows negative indices; using the bitwise NOT operator (~) provides a zero‑based reverse index.

print(a[-1], a[-2], a[-3])
print(a[~0], a[~1], a[~2])

2. Common Tools

2.1 Reading and writing CSV files

import csv
# Without header
with open(name, 'rt', encoding='utf-8', newline='') as f:
    for row in csv.reader(f):
        print(row[0], row[1])
with open(name, mode='wt') as f:
    f_csv = csv.writer(f)
    f_csv.writerow(['symbol', 'change'])

# With header
with open(name, mode='rt', newline='') as f:
    for row in csv.DictReader(f):
        print(row['symbol'], row['change'])
with open(name, mode='wt') as f:
    header = ['symbol', 'change']
    f_csv = csv.DictWriter(f, header)
    f_csv.writeheader()
    f_csv.writerow({'symbol': xx, 'change': xx})

If a CSV file is very large, increase the field size limit:

import sys
csv.field_size_limit(sys.maxsize)

CSV can also read tab‑separated values:

f = csv.reader(f, delimiter='\t')

2.2 itertools utilities

import itertools
itertools.islice(iterable, start=None, stop, step=None)
itertools.filterfalse(predicate, iterable)
itertools.takewhile(predicate, iterable)
itertools.dropwhile(predicate, iterable)
itertools.compress(iterable, selectors)

Sorting and grouping:

sorted(iterable, key=None, reverse=False)
itertools.groupby(iterable, key=None)
itertools.permutations(iterable, r=None)
itertools.combinations(iterable, r=None)
itertools.combinations_with_replacement(...)

Merging multiple sequences:

itertools.chain(*iterables)
import heapq
heapq.merge(*iterables, key=None, reverse=False)
zip(*iterables)
itertools.zip_longest(*iterables, fillvalue=None)

2.3 Counter

import collections
counter = collections.Counter(iterable)
counter[key]                 # frequency of key
counter.most_common(n=None) # top‑n most common elements
counter.update(iterable)
counter1 + counter2
counter1 - counter2
collections.Counter(list1) == collections.Counter(list2)

2.4 defaultdict with default values

import collections
dd = collections.defaultdict(type)  # type() is called to create a default value on first access

2.5 OrderedDict

import collections
od = collections.OrderedDict(items=None)  # preserves insertion order during iteration

3. High‑Performance Programming and Debugging

3.1 Writing errors and warnings

import sys
sys.stderr.write('error message')

import warnings
warnings.warn(message, category=UserWarning)
# categories: DeprecationWarning, SyntaxWarning, RuntimeWarning, ResourceWarning, FutureWarning

Control warning output from the command line:

$ python -W all     # show all warnings
$ python -W ignore  # ignore all warnings
$ python -W error   # turn warnings into exceptions

3.2 In‑code testing

# Debug block
if __debug__:
    pass

Running Python with the -O flag removes the block:

$ python -O main.py

3.3 Code style checking

pylint main.py

3.4 Measuring execution time

$ python -m cProfile main.py

# Timing a code block
from contextlib import contextmanager
from time import perf_counter

@contextmanager
def timeblock(label):
    tic = perf_counter()
    try:
        yield
    finally:
        toc = perf_counter()
        print('%s : %s' % (label, toc - tic))

with timeblock('counting'):
    pass

Performance‑optimisation principles:

Focus on actual bottlenecks, not the whole program.

Avoid global variables; local look‑ups are faster.

Prefer direct imports over attribute access (e.g., from module import name ).

Use built‑in data structures (str, list, set, dict) which are implemented in C.

Skip unnecessary intermediate objects and deep copies.

Prefer ''.join([...]) over repeated string concatenation; also consider using print(..., sep=':') instead of building a single string.

4. Other Python Tricks

4.1 argmin and argmax

items = [2, 1, 3, 4]
argmin = min(range(len(items)), key=items.__getitem__)
# argmax is analogous

4.2 Transposing a 2‑D list

A = [['a11', 'a12'], ['a21', 'a22'], ['a31', 'a32']]
A_transpose = list(zip(*A))               # list of tuples
A_transpose = [list(col) for col in zip(*A)]  # list of lists

4.3 Converting a 1‑D list to a 2‑D list

A = [1, 2, 3, 4, 5, 6]
# Preferred method
list(zip(*[iter(A)] * 2))