Distributed Real-Time Local Cache Practice in iQIYI TV Backend

High‑concurrency systems rely heavily on caching. Using more local caches can improve throughput and stability, but the main challenge is ensuring real‑time consistency of distributed local cache data.

This article details iQIYI TV’s backend distributed real‑time local cache solution as a reference for solving high‑concurrency problems.

Background

Most Internet services are read‑heavy. To improve stability and throughput, read and write paths are often split, and the cache discussed here targets the read side. iQIYI stores large amounts of metadata (several KB to tens of KB per item) that grow continuously. A dedicated micro‑service assembles this base data for downstream services.

The service faces extreme read pressure (hundreds of thousands of QPS) on a centralized cache, causing bandwidth loss and timeout issues.

Comparison of Local Cache vs. Centralized Cache

Local Cache Advantages

Hotspot caching – each new instance adds a hotspot DB.

High hit rate.

Custom expiration strategies.

Fast business logic, low machine wear.

Strong risk resistance.

Local Cache Disadvantages

Passive caching, poor real‑time freshness.

Limited storage (2‑4 GB), sufficient for most hotspot data.

Centralized Cache Advantages

Easy real‑time updates.

Strong consistency.

Centralized Cache Disadvantages

Large clusters create heavy dependencies.

IO bottlenecks under high concurrency.

Network jitter between application and cache nodes.

Hotkey traffic can saturate bandwidth, requiring multiple cache clusters.

Given these trade‑offs, most teams choose local hotspot caches, but real‑time freshness remains a problem. The proposed solution addresses this.

Solution Overview

A unified messaging mechanism triggers real‑time updates of local caches. Business services can filter messages to implement personalized update logic.

Solution Components

Management Console : Manages all application instances using the local cache and their caching policies.

Data Change Source : Origin of data modifications.

Message Bus : Central hub for distributing update messages.

Business Filter : Allows downstream services to process only relevant messages.

Monitoring & Statistics : Uses iQIYI’s unified logging system to collect cache hit metrics and support hotspot analysis.

Extension

If a single cluster’s local cache hit rate falls below an acceptable threshold (e.g., 70%) and memory cannot be expanded, the cache can be split into lightweight logical shards to improve hit rate.

Effect Summary

Reduced risk of cluster avalanche.

Resolved high‑concurrency read bottlenecks.

Decreased network penetration of hotspot data, easing the load on centralized caches.