Performance Optimization Techniques: Indexing, Caching, Compression, Prefetching, Throttling, and Batch Processing

Introduction: Trade‑offs

Software design is an art of taking and giving; achieving high performance often requires extra storage or complexity, and may conflict with security, scalability, or observability. The goal is to optimize the system before bottlenecks appear using common techniques.

Indexing Techniques

Indexes trade extra space for faster lookups, reducing time complexity from O(n) to O(log n) or O(1). They are widely used in databases and invisible in many front‑end/back‑end scenarios.

Hash Table – O(1) lookups, used by many language Map/Dict structures.

Binary Search Tree (e.g., Red‑Black Tree) – balanced tree used in Java TreeMap, Linux scheduler.

B‑Tree – multi‑way tree used by MongoDB indexes.

B+ Tree – leaf‑only storage, used by MySQL and many file systems.

LSM Tree – log‑structured merge tree, optimizes write performance in NoSQL stores.

Trie (Prefix Tree) – efficient prefix queries, used for autocomplete, URL routing.

Skip List – layered ordered list, used by Redis ZSet.

Inverted Index – keyword‑to‑document mapping, core of Elasticsearch and Prometheus.

When defining primary keys, consider auto‑increment IDs versus UUIDs; a hybrid approach such as Snowflake IDs often provides a good balance.

Caching Techniques

Caching also exchanges space for time. Multiple layers of cache exist from DNS, OS, CDN, server memory, page cache, SSD cache, to CPU caches.

DNS cache at browser, OS, and upstream servers.

Server‑side KV cache (e.g., Redis, Memcached).

Database page cache and OS page cache.

Hardware cache in SSDs and CPU.

Application‑level template cache.

Cache invalidation is one of the two hard problems in computer science, leading to issues such as cache penetration, cache breakdown, and cache avalanche, which require strategies like empty‑value caching, Bloom filters, request coalescing, and random TTL.

Compression Techniques

Compression trades CPU time for reduced data size. Lossless compression is used for HTTP responses (gzip/deflate), HTTP/2 header compression (HPACK), JS/CSS minification, binary RPC payloads, and storage of large objects. Lossy compression is common for media (video, audio, images) and for reducing bandwidth.

HTTP Accept‑Encoding gzip/deflate.

HTTP/2 HPACK header compression.

JS/CSS minification (Uglify, Terser).

Binary RPC and message‑queue compression (Snappy, LZ4).

Object pointer compression in JVM (UseCompressedOops).

BSON vs JSON storage in MongoDB.

Beyond compression, reducing data size by eliminating unused code, shrinking payloads, and using bitmaps or bit operations can further improve performance.

Prefetching

Prefetching extends caching by loading data before it is needed, reducing the perceived latency of the first access. Common scenarios include video buffering, HTTP/2 server push, client‑side pre‑loading, and server‑side hot‑data warm‑up.

Throttling (削峰填谷)

Throttling smooths traffic spikes by limiting request rates, using techniques such as leaky bucket, token bucket, debounce, and back‑off retries. Message queues (Kafka, RocketMQ, Redis) can buffer bursts, while scheduled tasks should be staggered to avoid “hair‑pin” spikes.

Batch Processing

Batching aggregates multiple operations into a single request, reducing per‑operation overhead. Examples include bundling JS files, sprite images, batch DB inserts, Redis MGET/MSET, and bulk writes in LSM‑tree based stores.

Front‑end: bundle JS, CSS, sprites.

Back‑end: batch DB inserts (5 000–10 000 rows), Redis MGET (50‑100 keys), message‑queue batch publishing (≤1 MB).

System I/O: write buffers, OS writev, LSM‑tree compaction.

Choosing the optimal batch size requires benchmarking in the target environment.

Advanced Topics (Mid‑ and Lower‑Part)

Focus : Reduce system‑call and context‑switch overhead, use epoll/event‑driven I/O, DMA and zero‑copy, and set CPU affinity.

Transformation : Adopt efficient data structures and algorithms (Fisher‑Yates, Dijkstra), replace List scans with Maps, minimize lock scope, and leverage JIT‑friendly code.

Adaptation : Optimize for specific runtimes – browsers (reduce reflow/repaint, use virtual DOM), Node.js (event loop tuning), Java (C1/C2 JIT, stack allocation), Linux (sysctl tuning, NUMA awareness).

Planning : Choose the right hardware (appropriate cloud instance types, ARM vs x86, bare‑metal), identify bottlenecks (CPU, memory, network), and consider specialized accelerators (GPU, AES‑NI, FPGA) only when ROI justifies.

There are only two hard things in Computer Science: cache invalidation and naming things.

Ultimately, performance optimization should be driven by measurement, profiling, and cost‑benefit analysis, avoiding premature or excessive tuning.

Latency Numbers Every Programmer Should Know