Master Python Backend Essentials: Generators, GIL, Django, and Linux Ops

Big Data File Reading

Use generators or simple iteration ( for line in file) to read large files efficiently.

Iterator vs Generator

Iterators are abstract objects that implement __iter__ and next; they raise StopIteration when exhausted. Generators are a concise way to create iterators using yield, automatically providing __iter__ and next, saving memory compared to list comprehensions.

Decorator Use Cases

Logging

Function execution time measurement

Pre‑processing before a function runs

Post‑processing after a function finishes

Permission checks

Caching

Global Interpreter Lock (GIL)

The CPython interpreter runs only one thread at a time, enforced by the GIL. In multithreaded scenarios the interpreter acquires the GIL, executes a slice of bytecode or yields control (e.g., time.sleep(0)), then releases the lock and repeats.

When calling external C/C++ extensions, the GIL is held for the duration of the call, preventing thread switches.

find and grep

grep searches text using regular expressions and prints matching lines. find locates files or directories that meet specified criteria, such as name patterns or ownership.

Linux Process Management

Supervisor is a powerful tool for managing background processes; after modifying files, restart with service supervisord restart.

Improving Python Performance

Use generators for lazy evaluation.

Leverage external packages (Cython, PyPy, etc.) for critical code.

Avoid attribute lookups inside tight loops.

Common Linux Commands

ls, help, cd, more, clear, mkdir, pwd, rm, grep, find, mv, su, date.

Yield in Python

Yield turns a function into a generator, remembering its state between calls so that each next() resumes execution where it left off.

Data Structures Comparison

Arrays store elements in contiguous memory; linked lists use non‑contiguous nodes. Queues follow FIFO order, stacks follow LIFO; both can be implemented with arrays or linked lists.

Sorting Algorithms

Various sorting methods exist; the most familiar one can be described in detail (e.g., quicksort, mergesort, bubble sort).

Python Memory Management

Garbage Collection : Python determines object types and memory at runtime; objects are reclaimed when no references remain.

Reference Counting : Each object tracks how many references point to it; when the count drops to zero, the object is freed.

Memory Pools : Python maintains several layers—C‑level malloc/free, internal pools for objects < 256 KB, and the top layer for Python objects. Small allocations are served from the pool to reduce fragmentation and improve performance.

Web Framework – Django

Session sharing across multiple app servers requires a common session store; otherwise login state is lost.

Cross‑origin requests are handled by enabling CORS middleware and adding {% csrf_token %} for POST forms.

Django follows the MVT pattern: Model (data handling with ORM), View (request processing), Template (HTML rendering).

Query ordering uses order_by(); prepend - for descending order; filter with field__gt=value for greater‑than queries.

Middleware sits between request and response, allowing global processing.

Django’s built‑in admin is generated automatically from ORM models, offering rapid CRUD interfaces.

Redirects are performed with HttpResponseRedirect (status codes 302 or 301).

Nginx Proxy Types

Forward proxy sits between client and origin server, requiring client configuration. Reverse proxy appears as the origin server to clients, forwarding requests to backend servers transparently.

Tornado Core

The core consists of IOLoop (event loop) and IOStream (non‑blocking socket wrapper), enabling high‑performance asynchronous I/O.

Why Not Use Django’s runserver in Production?

runserver uses Django’s built‑in WSGI server, which is single‑process and intended for development only.

uWSGI

uWSGI implements the WSGI, uwsgi, and HTTP protocols, offering fast performance, low memory usage, and multi‑app management; typically paired with Nginx for production deployments.