Recap of Shenzhen Elasticsearch Meetup – Community Growth, Compression Optimization, Real‑time Data Fusion, and Cluster Practices

The ES Chinese community, together with Tencent Cloud, held its first Shenzhen meetup on August 21, 2021. The event combined on‑site attendance, Tencent Meeting, and live streaming, featuring experts from Tencent, Tapdata, ByteDance, and Vivo.

Community Development : The meetup opened with a review of Shenzhen meetup history and an update on the ES community’s rapid growth – over 830 million total downloads of Elasticsearch, more than 2.5 billion downloads of the Elastic Stack, and a global community of over 100 k users. Nine Chinese cities now host regular meetups.

Talk 1 – Tencent Elasticsearch Compression Encoding Optimization : Engineer Bi Jieshan explained the challenges of Lucene’s storage engine, the high cost of storing multiple data structures, and the need for compression‑encoding improvements. He discussed the use of generic algorithms (Zstd, Brotli, Deflate) and custom encoding for specific structures, aiming to balance compression ratio and performance.

Talk 2 – ES + MongoDB Real‑time Data Fusion Platform : Architect Yang Qinglin (Tapdata) described a cross‑region data‑fusion solution built on Elasticsearch and MongoDB to address data consistency and latency for multi‑region retail. The platform replaces legacy Oracle systems, reduces maintenance costs, and supports high‑concurrency updates (MongoDB) and high‑concurrency search queries (Elasticsearch).

Talk 3 – ByteDance Elasticsearch Kernel Extensions : Engineer Lu Yuncheng presented ByteDance’s custom NFS‑based storage‑compute separation, multi‑copy cloud storage, and a leader‑to‑follower push replication model for near‑real‑time replica expansion. Additional extensions include vector‑search plugins, storage encryption, GDPR‑compliant access control, and detailed request‑level tracing for performance analysis.

Talk 4 – Vivo Elasticsearch Cluster Practices : DBA Liu Shilin shared Vivo’s large‑scale Elasticsearch operations, covering challenges such as high‑concurrency read/write costs, bug handling, and the use of an internally developed ES management platform. Topics included shard sizing (soft limit 1 000 per node, 30 000+ cluster‑wide), index lifecycle management via custom APIs, and scaling strategies without changing primary shard counts.

Each session included a Q&A segment covering version specifics, row‑store compression, storage‑compute separation, vector‑search performance, audit overhead, visualisation tools, and shard management.

The meetup concluded with a tea‑break networking session and a reminder that, despite pandemic constraints, the ES community continues to evolve and looks forward to the next Shenzhen meetup.