Optimizing JD Service Market's Plugin List Service: From Thread Scaling to a Unified Cache Layer

JD Service Market is an online transaction platform for JD merchants and third‑party independent software vendors (ISVs). As the platform grew rapidly, its architecture evolved from an ALL‑IN‑ONE design to a Service‑Oriented Architecture (SOA) to handle exponential traffic growth.

The performance of the entire market is limited by its weakest service; the "Available Plugin List" service is a core component whose optimization drives overall system improvements.

Optimizing the Service Itself

Upgrading from single‑threaded to multi‑threaded execution attempted to increase parallelism, but log analysis showed that the "service detail" step still dominated response time, indicating that external dependencies were the main bottleneck.

Resolving External Dependency Conflicts

Investigation revealed that the "Product Service" shared the same MySQL database with the "Category Service", causing resource contention under high concurrency. Separating these resources eliminated the conflict.

Further analysis showed that the Product Service itself monopolized database resources across its multiple functions. The solution was to decouple external dependencies based on usage scenarios:

Transactional consistency: continue using MySQL (InnoDB) for strong consistency.

Simple key‑value queries for user login data: use Redis.

High‑traffic, read‑only content: use Elasticsearch (ES).

Push notifications to external systems: use a message queue (MQ).

Complex analytics with relaxed real‑time requirements: use a big‑data platform.

Establishing a Unified In‑Memory Cache Model

Time‑space trade‑offs are fundamental; caching query results in memory (e.g., MySQL buffer pool) speeds up repeated reads. To handle larger data sets, Redis and ES were employed for different scenarios.

Choosing Active Caching Over Passive Caching

Passive caching with periodic expiration failed during peak hours because the cache was empty at 8 am, causing all requests to hit the database and leading to cache staleness during updates. The new approach proactively updates Redis after data changes and removes expiration, ensuring fresh data without unnecessary load.

Addressing Cache Fragmentation

Over time, many ad‑hoc caches were created for various external systems, resulting in low utilization and high maintenance overhead. By consolidating caches per data table and allowing some controlled redundancy, the system reduced fragmentation and improved performance.

Cache Construction Strategy

Challenges included asynchronous cache building, avoiding operation reordering, and ensuring transactional consistency. The solution leveraged JD's Binlake system to capture MySQL binlog events, publish them via MQ, and trigger cache updates based on the latest database state, eliminating reliance on message ordering and ensuring data consistency.

To further guarantee correctness, a periodic hourly comparison between source data and cache detects and compensates any discrepancies.

Post‑Implementation Results

After the refactor, the "Available Plugin List" service achieved significantly higher concurrency. During the 2018 Singles' Day (11.11) event, the service handled a six‑fold surge in calls within ten minutes without degradation.