Tencent's 4 Kernel Optimizations for Trillion-Scale Elasticsearch

Tencent Cloud Monitoring ingests tens of millions of writes per second per cluster, accumulating 14 PB over six months — equivalent to 1,500 physical machines — while maintaining four-nines availability. The team achieved this not by simply adding hardware but through four deep kernel-level optimizations.

1. Memory-Based Smooth Rate Limiting to Prevent Cluster Avalanches

Problem: A log cluster experienced a 5× write surge. Coordinator nodes ran out of heap memory during deserialization of incoming requests, triggering prolonged Old GC pauses that ejected multiple nodes and turned the cluster RED.

Root cause: Native Elasticsearch rate limiting uses a request-count leaky bucket that ignores actual memory pressure.

Solution: Treat JVM heap as the leaky bucket. Divide memory usage into zones and apply a cosine transform for smooth throttling: as memory rises, acceptance rate drops gradually rather than cliff-edge rejection.

Results: Availability improved from three to four nines; heap utilization increased 30%; nodes no longer OOM under extreme load.

2. Rollup Pre-Aggregation Slashes Storage Cost 10×

Problem: 14 PB on 1,500 machines was cost-prohibitive.

Analysis: Access frequency drops sharply after 5 minutes; data older than one month is used only for statistical analysis.

Solution: Hot/warm/cold tiering plus Rollup:

Hot data on SSD, warm on HDD, cold backed to object storage (~¥0.1/GB/month).

Rollup pre-aggregates second-level metrics into hourly/daily granularity. A 30-day metric that filled disks at second resolution shrinks to 13 GB at daily resolution.

Results: Cluster reduced to 150 machines (10× cost reduction); Rollup overhead stays under 10% of resources.

3. Off-Heap FST Enables 50TB Per Node

Problem: The Finite State Transducer (FST) in Lucene's inverted index resides permanently in heap, consuming 10–15 GB per 10 TB of disk. Heap usage exceeded 70%, causing frequent GC spikes and query latency jitter.

Three-step evolution:

Move FST off-heap using MMAP, but OS page reclaim caused severe read-performance regression.

Rewrite Lucene's buffer to store off-heap pointers enabling zero-copy reads; overhead dropped to 7%.

Add a two-level cache with Java weak references and a ReferenceQueue for automatic off-heap cleanup; performance matches native on-heap FST.

Results: Heap usage fell from 70% to 30%; single-node disk capacity scaled from 5 TB to 50 TB (10×).

4. Metadata Scalability Overhaul for Million Shards

Problem: Native master node performs multiple bidirectional metadata conversions and full-cluster sync on every index creation. At 30–50k shards and 500 nodes, index creation took 30 seconds to minutes.

Three improvements:

Task-based directed dispatch — avoid full-cluster broadcast.

Incremental metadata maintenance — eliminate full traversal.

Statistics caching — reduce redundant computation.

Results: Shard count scales to millions; node count to thousands; new index creation stabilizes under 5 seconds.

Reusable Four-Step Framework for Large-Scale ES

The optimizations form a prioritized framework:

Survive (availability) — memory-based smooth rate limiting.

Economize (cost) — tiered storage + Rollup pre-aggregation.

Accelerate (performance) — time-based tiered merge + Composite aggregation jump (query latency improved 40%).

Scale (scalability) — incremental metadata + task-based dispatch.

The sequence matters: stay alive first, then reduce cost, then improve speed, finally expand capacity.

"Tuning cannot solve the problem; you must modify the kernel." — Tencent's 20+ production PRs prove Elasticsearch's ceiling is far higher than assumed.