JD’s Color Gateway: Tens of Millions QPS with Cloud‑Native Data Warehouse

What is the Color Gateway?

Before introducing the Color gateway, we first explain what an API gateway is. An API gateway receives all client requests, routes them to backend services, provides aggregation and protocol conversion, and can aggregate results from multiple services, reducing integration cost, improving efficiency, and ensuring security.

The JD Retail Color gateway, as an API gateway service, handles hundreds of billions of daily traffic and calls, and is a crucial technical component for promotional events.

Challenges During Major Promotions

The Color gateway faces four main challenges during large‑scale sales events:

Stability: must ensure smooth operation under high‑concurrency traffic from core services.

High availability: thousands of backend interfaces must operate independently so that a failure in one does not affect others.

Reliability: security measures such as anti‑scraping and flow control are needed to prevent overload.

Performance: real‑time monitoring of gateway performance and request interception is essential for decision‑making.

Architecture Selection and Solution

1. Original Architecture – HBase Pain Points

The monitoring service, core of the gateway, relied on HBase for log analysis, but HBase exhibited several shortcomings:

Inability to meet multi‑dimensional aggregation query performance requirements.

Lack of time‑series aggregation capability (requiring manual development).

Hotspot and memory consumption issues.

Limited data analysis performance, unable to meet response time expectations.

To address these issues, the architecture was upgraded.

2. Upgrade – Cloud‑Native Data Warehouse Solution

JD Cloud developed a cloud‑native data warehouse based on ClickHouse, separating storage and compute.

The upgraded architecture divides traffic governance into two services: full‑data (offline, written to reports and monitoring logs) and real‑time data (streamed via Kafka to Flink, then to the cloud‑native warehouse for real‑time analysis).

Compared with HBase, the cloud‑native warehouse improves processing performance by 60% (from 1.92 × 10⁸ rows/min to 3.1 × 10⁸ rows/min) and enables second‑level scaling of compute resources, dramatically reducing storage costs.

3. Problems with Traditional Data Warehouses

Traditional warehouses suffer from technical bottlenecks, complex architecture, high query latency, difficulty achieving high concurrency and availability, and high operational costs.

4. Required OLAP Engine for Explosive Business Growth

To meet massive data demands, an OLAP engine with the following characteristics is needed:

Separation of storage and compute to cut storage costs by over 50% and allow rapid elastic scaling.

High availability and ultra‑high performance, delivering sub‑second response even under massive load.

Out‑of‑the‑box usability: one‑click cluster creation, visual monitoring, and immediate production readiness.

5. Advantages of the Cloud‑Native ClickHouse Warehouse

Key benefits include:

Very low cost: storage cost reduced by more than 80% per PB.

Extreme elasticity: compute nodes can be scaled up or down within seconds.

Simplicity: one‑click provisioning of PB‑scale instances with visual monitoring and log analysis.

6. ClickHouse Architecture

The system adopts a storage‑compute separation at the database management layer, with a cloud‑control platform providing user‑facing operations such as purchase, configuration, and instance management.

7. Compute and Storage Node Advantages

Compute nodes are stateless, focusing solely on dynamic resource allocation, improving utilization. Storage nodes are also stateless, organized into groups that handle concurrent requests and support multi‑level caching, enabling fast data access and easy scaling when I/O becomes a bottleneck.

8. Second‑Level Elasticity

Users do not need to manage data distribution; they only need to consider compute capacity, while the system automatically handles storage scaling.

9. Cost Reduction

ClickHouse’s multi‑replica shared storage cuts storage cost by 50%, distributed storage pricing is about 60% of traditional cloud disks, and billing is based on actual data volume (50‑80% of raw size), yielding overall savings of 50‑80%.

10. Supporting JD Business

The cloud‑native warehouse now powers dozens of core JD services, including real‑time dashboards, promotional product selection platforms, precise marketing, advertising, log and traffic analysis, and more. Its elastic architecture allows dynamic scaling during peak and off‑peak periods, letting users focus on business value.