JD.com’s Big Data System Upgrade for the 11.11 Shopping Festival: Multi‑Region Scheduler, Intelligent Storage, Containerized Streaming, and Blockchain Traceability

“Technological innovation must serve business model innovation; the most valuable tech breakthroughs are engineered applications that can be generalized and scaled.” This principle guides JD.com’s recent big‑data system upgrade ahead of the 11.11 Global Shopping Festival.

The upgrade focuses on three core areas: a multi‑region Hydra Scheduler, an intelligent storage strategy, and a fully containerized streaming computation platform. By separating storage and compute, enhancing cross‑region scheduling, and applying real‑time node health metrics, the scheduler now supports over ten thousand machines and delivers a 4‑5× boost in task allocation speed.

The intelligent storage policy continuously monitors CPU, memory, disk, and network load to place replicas on less‑busy nodes, reducing I/O contention by 300% and improving overall cluster balance.

Containerizing the streaming platform enables mixed CPU‑ and memory‑intensive workloads, achieving roughly 30% higher resource utilization and simplifying global roll‑outs via single‑image deployments.

Log reporting was also overhauled: a differentiated strategy now considers log type, app version, device ID, and event ID, allowing either time‑interval or count‑based triggers. This change cut log latency, increased server request handling by 3×, and doubled log‑processing capacity, achieving over 95% of logs reported within one minute.

For the JD Zhizhen Chain anti‑counterfeit traceability platform, data was sharded by supplier, migrated to a distributed elastic database, and cached for two hours, while blockchain optimizations raised transaction throughput five‑fold. These steps reduced QR‑code response time from an average of 1 second to 150 ms.

Overall, the upgrades delivered a three‑fold reduction in core task completion time, a 30% saving in container resource usage, and a markedly smoother consumer experience during peak traffic periods.