Design Practices for Large‑Scale Microservice Frameworks

Du Huan, a senior expert engineer at Didi, gave a deep technical talk titled “Large‑Scale Microservice Framework Design Practice” at the Go China conference. The talk is organized into several parts: problem discovery, historical evolution of service frameworks, design principles, key implementation details, business benefits, version management, and future work.

Problem discovery – pain points in service development

The speaker listed typical difficulties in complex business development: tight schedules, large teams, rapid business growth, uneven developer skill levels, and the need to integrate many internal tools, SDKs, and best‑practice guidelines. He emphasized that fast‑moving internet services often suffer from “quick, rough, fierce” development, and that a solid architectural foundation is needed to keep quality stable.

Historical perspective – evolution of service frameworks

He traced the history from early PHP (1995) and ASP.NET (2002) to the rise of MVC frameworks (Django, Ruby on Rails) and the explosion of MVC‑style frameworks after 2005. He highlighted the shift toward routing and RPC frameworks (Sinatra, Thrift, Go martini) and the impact of containerization (Docker) and service‑mesh solutions like Istio.

Design goals – “large‑scale microservice framework”

The framework aims to act as an operating‑system‑like layer for services, providing unified governance, horizontal scaling, and seamless integration with existing infrastructure. It follows the “Rule of Least Power” principle: only expose the essential, stable abstractions and keep the design minimal.

Key implementation details

Unified interface layer that abstracts all I/O (RPC, storage, etc.) behind Go interfaces, allowing the business code to remain unchanged when underlying drivers evolve.

Automatic protocol hijacking using a finite‑state‑machine wrapper around Thrift/HTTP to inject logging, tracing, and context propagation without touching business logic.

Low‑precision timer pool to avoid excessive timer and channel allocations, reducing overhead while meeting millisecond‑level timeout requirements.

Fail‑fast and circuit‑breaker mechanisms that propagate upstream timeout budgets downstream, preventing cascade failures.

Extensive use of reflection and AST generation to auto‑generate routing code from IDL definitions, enabling rapid service refactoring.

Business impact

The framework has been deployed in a massive Didi business system with thousands of services and millions of lines of Go code. Benefits reported include:

Significant improvement in development efficiency and system stability.

Transparent upgrades: developers only need to pull the latest framework version; compatibility is guaranteed by strict interface stability.

Reduced operational incidents: automatic retries, connection leak fixes, and unified timeout handling dramatically lowered the occurrence of snowballing failures.

Version management and future work

The team treats the framework like an operating system: no per‑library tags, always upgraded to the latest version to avoid version fragmentation. Future plans involve expanding toolchains, deeper integration with internal infrastructure, and possibly open‑sourcing the design concepts.

Q&A highlights

Go services run as single processes; GOMAXPROCS controls CPU usage.

Service interfaces are managed via IDL; compatibility is the responsibility of the business team.

Timeouts are set per request, propagated downstream via context, and currently managed manually by developers.