Mastering System Design: Real-World Lessons from Alibaba’s Architecture Veteran

Alibaba senior technical expert Bi Xuan, who joined the company in 2007 and helped build HSF, reflects on more than a decade of experience in foundational technologies and explains why system design is far more challenging than Java programming and often becomes overly theoretical.

He launched an internal, informal training program to help architects grasp a practical system‑design framework.

Training Goals

Provide a thinking framework that reveals the systematic steps of system design, emphasizing that design must follow a disciplined process rather than ad‑hoc sketching.

Broaden participants’ knowledge to consider comprehensive trade‑offs, enabling better decision‑making during design.

System‑Design Routine

The routine consists of five stages: purpose → goals → core design around the goals → design principles derived from the core design → detailed design of each subsystem or module.

1) Purpose of System Design

The purpose clarifies why a new system or a major refactor is needed; without a clear purpose, later stages easily drift, leading to solutions that do not address the original business challenge.

2) Goals of System Design

Goals translate the purpose into measurable targets, ensuring the final implementation aligns with the original intent and providing a way to track success.

3) Core Design Around the Goals

This stage defines how the system will achieve the goals, balancing technical choices, architectural vision, and trade‑offs to produce a concrete core design.

4) Design Principles Derived from the Core Design

Principles guarantee consistency across detailed subsystem designs, ensuring the overall architecture remains coherent.

5) Detailed Design of Subsystems/Modules

With the groundwork laid, engineers focus on solving smaller, well‑defined problems within each module, leveraging solid mathematical and problem‑solving skills.

Case Studies

HSF Design

HSF aimed to build an easy‑to‑use RPC framework capable of handling billions of daily calls. Early versions suffered from poor load‑balancing, inadequate monitoring, and insufficient versioning, leading to repeated refactors. Lessons learned include the importance of deep familiarity with chosen components, rigorous performance testing, and designing for observability.

T4 Design

T4 focused on containerizing applications to run multiple workloads on a single machine. Initial hacks achieved limited success, but the adoption of LXC provided a robust solution. The project highlighted the need for broad technical vision when selecting underlying technologies.

Multi‑Active Deployment Design

The multi‑active design tackled two core problems: traffic isolation and data consistency across geographically distributed sites. Design decisions covered traffic splitting rules, database sharding strategies, synchronization approaches, CAP trade‑offs, deployment topology, and rollout cadence.

Unified Scheduling Design

The unified scheduler aimed to allocate resources for both online services and offline tasks, addressing challenges such as resource contention, expanding the resource pool, and ensuring interoperability between the two scheduling domains.

Key Capabilities for Architects

Understanding business challenges and mapping them to technical problems.

Comprehensive knowledge covering development, deployment, operation, and maintenance.

Strong technical foundation and broad vision for informed technology selection.

Ability to weigh trade‑offs under various constraints and establish guiding principles.

System design remains one of the most difficult topics to teach, but with practical training, real‑world case studies, and continuous reflection, architects can develop the skills needed to create robust, business‑aligned systems.