Performance Optimization Techniques: Indexing, Caching, Compression, Prefetch, Throttling, and Batch Processing

01. Introduction: Trade‑offs

Software design and development is an art of "taking" and "giving"; high performance often incurs higher implementation cost and may conflict with other quality attributes such as security, scalability, and observability. Before a business hits a bottleneck, we usually apply common technical measures to reach the expected performance level.

Performance optimization can be seen as a series of "time" and "space" trade‑offs. This article is divided into two parts. The first part covers six generic techniques that exchange time for space, and the second part introduces four advanced techniques mainly related to improving parallelism.

02. Indexing

Indexing works by using extra storage space to reduce query time, at the cost of additional write overhead. It can lower read complexity from O(n) to O(log n) or even O(1). Common index data structures include:

Hash Table : Provides O(1) lookup by mapping keys to hash values; used in many key‑value stores and language‑level maps/dictionaries.

Binary Search Tree (e.g., Red‑Black Tree) : Balanced tree used in Java TreeMap/TreeSet, Linux CPU scheduler, etc.

B‑Tree : Multi‑way balanced tree suitable for large datasets; used by MongoDB.

B+ Tree : Variant where only leaf nodes store data; used by MySQL and many file systems.

LSM Tree : Log‑Structured Merge Tree optimizes write performance at the expense of read speed; common in NoSQL stores.

Trie (Prefix Tree) : Enables fast prefix searches, auto‑completion, URL routing; radix tree variant used in Nginx and Redis.

Skip List : Multi‑level ordered list; suitable for high‑concurrency writes; used by Redis ZSet.

Inverted Index : Maps terms to the documents/pages they appear in; core to full‑text search engines like Elasticsearch and Prometheus.

When creating indexes, follow best practices: define primary keys (clustered indexes), add indexes on columns used in WHERE, GROUP BY, ORDER BY, JOIN, avoid indexing low‑cardinality columns, consider composite indexes to achieve index covering, reduce redundant indexes, and use database‑specific features such as TTL, sparse, or geo indexes.

03. Caching

Caching follows the same principle as indexing: trade extra storage for faster reads. Caches exist at many layers—from DNS caches, CDN edge caches, server‑side KV caches, database page caches, OS file caches, hardware disk caches, to application‑level caches such as template engines, browser caches, and CPU caches.

Typical cache‑related challenges include cache invalidation, cache penetration, cache breakdown, and cache avalanche. Solutions involve using empty‑value caching or Bloom filters, query mutexes, random TTLs, and careful multi‑level cache eviction strategies.

Object‑pooling (e.g., JVM constant pool, database connection pools, HTTP connection pools, thread pools, Go's sync.Pool) can also be viewed as a cache variant that reuses expensive resources.

04. Compression

Compression trades CPU time for reduced storage space. Lossless compression (e.g., Gzip, Deflate, HPACK, minification, binary RPC encoding) saves bandwidth and storage, while lossy compression (e.g., video/audio codecs, image JPEG) sacrifices fidelity for size.

Information theory tells us lossless compression is bounded by entropy; further reduction requires lossy methods. Additional space‑saving tactics include tree‑shaking unused code, enabling HTTP/2 and modern TLS, trimming unnecessary headers/cookies, using incremental updates (PATCH), shortening URLs, and employing bitmaps or bit‑level operations for massive flag data.

05. Prefetching

Prefetching builds on caching by spending time up‑front to fetch data that is likely to be needed soon, thereby reducing the perceived latency for the first request. Common scenarios include video buffering, HTTP/2 server push, client‑side pre‑loading, server‑side hot‑data warm‑up, and pre‑allocation of scarce resources such as inventory in flash‑sale systems.

Prefetching introduces side effects: longer startup time, extra resource consumption, and the risk of fetching data that may never be used.

06. Throttling and Smoothing (Peak‑Shaving)

Peak‑shaving trades time for time by smoothing traffic spikes. Techniques include front‑end lazy loading, back‑pressure control (rate limiting, debouncing), queuing bursts with message queues (Kafka, RocketMQ, Redis), deferring non‑critical scheduled tasks, and implementing exponential back‑off for retries.

07. Batch Processing

Batch processing also follows a time‑for‑time trade‑off: by grouping many operations into a single batch, overall throughput improves at the cost of occasional larger latency. Use cases span front‑end asset bundling, request aggregation (GraphQL), message‑queue bulk handling, bulk database inserts, and OS‑level buffered I/O (writev/readv).

Choosing the optimal batch size depends on the specific system: for Redis MGET/MSET, 50‑100 keys is typical; for MySQL bulk inserts, 5,000‑10,000 rows often yields best performance; for message queues, keep payloads under 1 MiB.

08. Summary

The first half of the article covered "time vs. space" trade‑offs that are broadly applicable, while the second half will dive into more advanced, less universally applicable performance‑optimization techniques.

There are only two hard things in Computer Science: cache invalidation and naming things.

Source: code2life.top/2020/08/15/