How JD.com Scales Its Homepage: Frontend Architecture, Static Rendering, and Engineering Practices

JD.com Homepage Frontend Architecture Design and Implementation

In March 2016, the Internet Technology Alliance (ITA1024) launched a Frontend Technology Month, featuring weekly online classes and monthly offline conferences where experts shared insights on frontend development, frameworks, performance tuning, and composite technologies.

Guest speaker: Liu Wei, Senior Frontend Architecture Engineer at JD.com.

Host: Zhao Xiaoqiang, Senior Frontend Engineer at Baidu.

1. Challenges

DOM elements increased dramatically (tab tags per floor from 5 to 9).

Overall page height doubled (14 floors, height from 4820px to 9862px).

Image count surged, with 80% of positions now displaying images.

First‑screen load time must not increase.

Homepage must remain error‑free and visually complete.

High traffic and concurrency demand extreme stability.

Frequent urgent business requirements from multiple stakeholders.

2. Frontend Page Staticization

Because the homepage receives massive traffic, querying the backend for each request would cause huge overhead. JD.com therefore generates static pages and serves them via a CDN.

Architecture:

Access layer: CDN → HAProxy → Nginx

Application layer: PHP, Redis

Storage layer: MySQL

Users request the homepage from the CDN, which delivers a pre‑assembled static file. Nginx caches the request, forwards it to PHP, which uses Redis for caching and MySQL for data. A scheduled task runs every two minutes to generate static files and distribute them to CDN nodes.

3. Overall Frontend Architecture

JS Layer

jQuery 1.6.4 is used for compatibility across thousands of pages.

JDF public components (UI, business widgets) provide common interactions such as lazy loading, carousel, region selector, dialogs, etc.

Page‑specific scripts render recommendation, guess‑you‑like, advertising, and discount sections.

CSS Layer

Base styles are decoupled into a lightweight ui-base.css; each component loads its own styles on demand.

All floor styles are combined into a single first‑screen request to ensure high availability.

4. Page Loading Strategy

Typical flow: backend assembles data, images are lazy‑loaded, and the browser loads all HTML elements.

JD.com adopts floor‑level asynchronous loading with local caching:

Each floor is a separate DIV with its own data API; the first two floors are pre‑loaded.

Floor data is cached in localStorage using an MD5 timestamp; identical timestamps skip network requests.

If the first AJAX request fails, a second request is attempted; if both fail, the floor is hidden to avoid blank space.

This reduces first‑screen elements to about 900 out of 3288 total, achieving ~1.6 s first‑screen load.

5. Basic Frontend Infrastructure

6. Frontend Tools and System (JDF)

JDF is a Node.js‑based command‑line tool that supports the full development workflow:

Create project scaffolding with JDFinit.

Compile modules using JDFbuild.

Output optimized assets with JDFoutput, automatically generating CDN‑prefixed URLs and combo requests.

Upload compiled files to test servers via JDF upload.

Inject analytics tracking points.

Integrate head/tail code from backend systems.

Deploy with Git and the online JDF platform, then clear CDN cache.

7. Disaster Recovery Strategy

Prefer local localStorage cache for async data.

Retry failed requests.

Use a dual‑layer approach: CDN first, then origin server.

Fallback to pre‑prepared backup data.

Disable non‑core interfaces when necessary.

8. Performance Optimization

Minimize first‑screen elements via async loading and caching.

Inline critical CSS to avoid blank pages when CDN is slow.

Combine CSS/JS requests and use sprites.

Compress static assets (CSS, JS, images).

DNS pre‑fetch for external domains.

Reduce cookie size.

Frontend Engineering Practices on an E‑Commerce Homepage

JDF provides a comprehensive solution covering command‑line tools, modules, development workflow, components, and documentation.

GitHub: https://github.com/putaoshu/jdf/

1. Command‑Line Tool

Cross‑platform (Windows, macOS, Linux).

Generates standardized project directories.

Supports local, integration, and production workflows.

Each project has a configuration file for unified compilation.

2. Frontend Module Definition

A widget consists of:

Folder: ui-product-list Configuration: component.json Data source: ui-product-list.json Template: ui-product-list.vm (or .smarty)

Style: ui-product-list.scss Script: ui-product-list.js Images folder.

Widget usage example: {%widgetname="ui-product-list"%} During build, JDF compiles the JSON data into the template, generates static HTML, and converts SCSS to CSS, inserting appropriate <link> and <script> tags.

Output combines multiple widget resources into a single request, e.g.:

??/widget/ui-product-list/ui-product-list.css,/widget/header/header.css,/widget/footer/footer.css

3. Module Usage Commands

jdf widget -install <name>

: install reusable widget. jdf widget -create <name>: create new widget. jdf widget -publish <name>: publish widget to the module cloud. jdf widget -preview <name>: preview widget locally. jdf widget -list: list all widgets.

4. Frontend Components

UI interaction components and business widgets are shared across the site, including lazy loading, carousels, region selectors, dialogs, etc.

5. Development Process

Initialize project with JDFinit.

Install needed widgets from the module cloud.

Compile with JDFbuild.

Output with JDFoutput.

Upload to test servers for integration.

Commit to Git and package with online JDF for production.

6. Documentation

Documentation covers standards, component APIs, and command‑line usage. See the GitHub repository for detailed docs.

Q&A

Q: Is the module cloud self‑hosted or based on a private npm registry?

A: It is self‑hosted on a Linux FTP server, with version control in component.json. Modules can be published and downloaded via the JDF CLI.

Q: What does the class attribute format class="xxx|yyy|zzz" mean?

A: It is the clstag front‑end tracking syntax, where xxx is the page, yyy the floor, and zzz the element identifier.

Q: What language is JDF written in?

A: Node.js.

Q: How is CDN cache cleared after a release?

A: By updating the version prefix in static resource URLs (e.g., product/home/1.0.0/a.js → product/home/1.0.1/a.js) or manually purging specific files via the CDN console.

Q: How does static‑page generation work?

A: Cache‑Control headers enforce a two‑minute update cycle; static fragments are generated on a dedicated server and synced to CDN nodes via Nginx scheduled tasks.

Q: How are personalized or flash‑sale sections handled?

A: Those sections are loaded asynchronously via AJAX, bypassing static HTML.

Q: How does JDF differ from tools like Gulp, Grunt, or FIS?

A: JDF is lightweight, relies on carefully selected Node.js plugins, and provides built‑in support for .smarty and .vm template rendering without requiring Java or PHP runtimes.

Q: How is CDN cache managed for updated files?

A: A unified version prefix is added to all static assets; for minor changes, the CDN cache can be manually purged.

Q: How are DNS pre‑fetches supported across browsers?

A: Supported in Chrome, IE 9+, Firefox 3+, Safari 5+, and improves performance when many external domains are used.

Source: http://www.cnblogs.com/xbcq/p/5362805.html