Tagged articles

metadata scalability

2 articles · Page 1 of 1

Aug 26, 2026 · Databases

Tencent's 4 Kernel Optimizations for Trillion-Scale Elasticsearch

Tencent shares four kernel-level Elasticsearch optimizations: memory-based smooth rate limiting to prevent cluster avalanches, Rollup pre-aggregation cutting storage costs 10x, moving FST off-heap to manage 50TB per node, and metadata scalability improvements supporting million shards with sub-5-second index creation.

ElasticsearchRollupTencent

0 likes · 7 min read

Tencent's 4 Kernel Optimizations for Trillion-Scale Elasticsearch

Didi Tech

Mar 28, 2019 · Big Data

Scaling Hive Metadata Storage with Federation Architecture

Didi solved Hive’s MySQL metadata bottleneck by building a federation architecture—using waggle_dance to route requests to multiple MySQL instances based on database names—enabling horizontal scaling, read/write support, and seamless compatibility with existing Hive clients while improving stability and performance.

Hive FederationMySQL optimizationWaggle Dance

0 likes · 7 min read

Scaling Hive Metadata Storage with Federation Architecture