How JD’s API Gateway Handles Tens of Millions of Concurrent Requests
This article explains JD's API gateway architecture and practices—including asynchronous high‑concurrency handling, fine‑grained security controls, gray‑release strategies, and automated operations—that enable the platform to efficiently serve millions of simultaneous requests during major shopping events.
Overview
JD’s e‑commerce platform has seen rapid growth in users and traffic, prompting the creation of an API gateway to unify client‑side access and simplify server‑side development.
Background
The gateway standardizes interfaces for clients, reduces integration costs, and shields backend services from public‑facing issues, improving development efficiency.
API Gateway Introduction
The gateway acts as a bridge between clients and services, handling massive concurrent requests, providing security protection, data monitoring, alarm mechanisms, multi‑protocol adaptation, and gray‑release capabilities.
Technical Practice
The gateway serves native clients, web, and mini‑programs. Its architecture consists of three layers: a VIP layer receiving HTTP/HTTPS requests, a gateway layer performing validation and routing, and the backend business API layer.
High Concurrency Implementation
To handle near‑ten‑million concurrent requests during major shopping festivals, the gateway adopts asynchronous processing with NIO multiplexing. Synchronous processing ties thread release to backend response time, causing thread‑pool exhaustion and low throughput. Asynchronous handling releases threads immediately, allowing the gateway to continue accepting new requests while waiting for backend responses.
Figures illustrate the synchronous and asynchronous processing flows.
Security Protection
Fine‑grained traffic control uses a token‑bucket algorithm with queuing or circuit‑breaker strategies. Authorization ensures only approved APIs are accessible. Signature authentication uses HMAC‑SHA256 to verify request integrity. Cross‑origin validation protects web and mini‑program clients from malicious attacks.
Gray Release
The gateway supports targeted gray testing by device ID or user ID and percentage‑based rollouts, allowing a small portion of traffic to be directed to new services for validation before full deployment.
Automated Operations
By platformizing the API gateway, multiple product lines share a single gateway, enabling automated deployment, rapid scaling, unified management, and sustainable product development. The console provides visual monitoring, real‑time analytics, and alarm subscription for API performance and fault diagnosis.
Summary
The API gateway serves as the sole entry point to backend services, offering componentization, structured demand handling, data‑driven configuration, and visualized business monitoring. Integrating a high‑performance, secure, and automated gateway allows microservice backends to focus on core business logic while benefiting from unified routing, authentication, and monitoring.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
ITFLY8 Architecture Home
ITFLY8 Architecture Home - focused on architecture knowledge sharing and exchange, covering project management and product design. Includes large-scale distributed website architecture (high performance, high availability, caching, message queues...), design patterns, architecture patterns, big data, project management (SCRUM, PMP, Prince2), product design, and more.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.