From Zero to Launch: Building JD.com’s UGC “Home Page Takeover” Project in 60 Days

JD.com aimed to transform its cold, shopping‑focused app into a warm, user‑generated‑content (UGC) platform that could surface popular posts on the home page, thereby increasing brand affinity and user engagement.

The team received the requirement in late October 2019 and had to deliver a fully tested, production‑ready system by the end of December, giving them only 60 days for development.

Architecture Overview

The solution was split into three layers:

Frontend: Handles data formatting and exchange with the client.

Middle‑platform (data hub): Stores data and provides RPC services.

CMS backend: Enables operators to manage posts, activities, and tasks.

The overall architecture diagram is shown below:

The frontend consists of two projects: an SOA service for the main app and an H5 project for WeChat sharing and user propagation. The middle platform uses JSF to expose RPC services, while the CMS backend communicates via JMQ messages, decoupling it from the middle‑platform logic. Data is stored in JimDB (a Redis‑compatible KV store) and a MySQL‑based elastic database (JED).

Technical Practices

3.1 JimDB Structured Data

Post data is stored as a Redis‑style zset for sorted lists (hot posts by likes, latest posts by timestamp) and as String entries for the post payload, serialized as JSON for readability.

The following table compares binary (protostuff) and JSON serialization:

优劣对比

序列化成二进制（protostuff）

序列化成JSON

序列化速度

快

相对慢

序列后大小

小

相对大

序列化后可读性

完全不可读

可读

Redis 中数据类型

String

String

Performance tests inserting 100,000 posts and querying 20 per page showed JimDB achieving roughly 6,000 QPS with a TP99 under 10 ms, compared to MySQL’s 3,000 QPS and 150 ms TP99.

3.2 JMQ Decoupling

To avoid tight coupling between the CMS backend and the middle platform, JMQ was introduced so the CMS can modify data via messages without needing direct access to JimDB.

A consistency issue was observed when the JED master‑slave replication lag caused stale reads; the recommendation is to perform critical reads/writes on the master.

3.3 Asynchronous and Parallel Network I/O

Using Java’s java.util.concurrent utilities, the team parallelized independent API calls and database queries. For example, the post list endpoint now batches user info queries to JimDB and concurrently fetches like information via CompletableFuture , reducing request latency from ~350 ms to ~40 ms.

Performance tables before and after the refactor illustrate the improvement:

状态

单机QPS

TP99

CPU 使用率

改造前

800 左右

350ms 左右

75% 左右

改造后

1200 左右

40ms 以内

85% 左右

Further testing showed CPU becoming the bottleneck on an 8‑core, 16 GB machine.

Summary & Outlook

The “Home Page Takeover” UGC project attracted over 12,000 user posts and 15 million views, demonstrating a successful shift in JD.com’s product strategy. Future work includes improving modularity and configurability, extracting reusable components such as task management, and continuing performance and feature enhancements.