Tencent Meeting "My Recordings" List Optimization Practice

This article presents a comprehensive design and optimization case study of the list interface used in Tencent Meeting’s “My Recordings” feature.

Background : List queries are a common backend service pattern, often coupled with pagination and sorting. When dealing with massive data volumes (hundreds of millions of records) and high concurrency (hundreds of QPS), naive implementations quickly become performance bottlenecks.

Key Challenges :

Large request volume and strict latency requirements (sub‑100 ms for first‑page loads).

Deep pagination leading to offset‑based scans that cause full table scans and exponential latency growth.

Multi‑source data aggregation (own recordings, recent views, authorized recordings, deletion records) which makes pagination and sorting across databases extremely complex.

Deep‑Pagination Solutions :

Replace offset‑based LIMIT X,30 with a WHERE id > last_id LIMIT 30 “seek method” to avoid scanning and discarding rows.

Use delayed‑join patterns to first fetch only primary keys and then join the detailed rows.

Consider cursor‑based pagination for ordered results, ensuring the ordering column is monotonic (e.g., using Snowflake IDs).

Cache Design :

Result‑level cache for the first page (list of IDs) – simple but suffers from consistency and cache‑bloat issues.

ID‑list + element cache – store only the IDs of a page, then fetch details from a separate element cache; improves consistency and reduces cache size.

Combined ID‑list + element cache – the final solution adopted, using Redis List/ZSet for ID pagination and a per‑record element cache for details.

Implementation details include a two‑layer cache for “My Recordings”: the first layer caches the ordered ID list for the first page, the second layer caches individual recording details. When a record changes, only the affected element cache entry is invalidated, while the ID list can be refreshed lazily.

Storage Selection for Multi‑Source “All Files” :

After evaluating MySQL, TDSQL, Redis, Elasticsearch, and MongoDB, MongoDB was chosen for its horizontal scalability, query performance, and moderate cost. The design uses a composite shard key {uid, record_id} to avoid data skew and ensure queries hit the shard key.

Data Consistency :

Reliably consume change events via Kafka (at‑least‑once) with dead‑letter handling and idempotent processing (Redis‑based idempotency keys).

Periodic reconciliation jobs compare source tables with the aggregated collection to detect missing or stale entries.

Only index fields are stored in the aggregated collection; mutable fields (title, status, permissions) are fetched on‑demand from the original services, eliminating the need to keep them in sync.

Performance Results :

After applying the two‑layer cache, the “My Recordings” API handled >700 QPS with average latency reduced from 308 ms to 70 ms (≈4× speed‑up).

For the “All Files” view, after full caching of individual records, latency dropped to ~96 ms at 700 QPS, meeting the sub‑500 ms requirement for a “second‑open” experience.

Takeaways :

Use cursor‑based pagination to avoid deep‑offset scans.

Cache IDs separately from record details to balance cache hit rate and consistency.

Select a storage engine that matches data volume, query patterns, and cost constraints (MongoDB in this case).

Combine reliable messaging with periodic reconciliation to guarantee data consistency across heterogeneous data sources.