Evolution of a Python Backend Architecture: From MVC to Microservices and Domain‑Driven Design

Before joining Tencent, the author spent three years as a backend developer on a human‑resource SaaS product, documenting the gradual evolution of its backend architecture and the problems encountered at each stage.

The product serves HR users via web, Android/iOS, and mini‑program clients, exposing RESTful APIs built primarily with Python for rapid iteration.

The architecture progressed through four major phases: 1) MVC, 2) Service Splitting, 3) Microservices, and 4) Domain‑Driven Design.

1. MVC

In the early stage, a small team (<5 developers) used Django to quickly prototype features, designing database tables first and abstracting functionality into Views. Emphasis was placed on coding standards and linting. The deployment stack included Nginx for load balancing, multiple Django instances, Celery for asynchronous tasks, Redis for caching, and Nginx Push Module for real‑time notifications.

Problems and optimizations:

Django’s concurrency was improved by running uWSGI with gevent workers.

Redis connection overload was mitigated using redis‑py’s built‑in connection pool.

MySQL connection pressure was reduced with djorm‑ext‑pool connection pooling.

Celery was configured to use gevent for concurrent task execution.

As the number of Django apps grew, deployment became cumbersome because each release required restarting all Django services, leading to downtime and overtime debugging.

2. Service Splitting

With a larger backend team, responsibilities were divided, and the monolithic codebase was split into separate services. Common code was extracted into a shared Python library, while the database and Redis remained shared initially, later scaling to multiple DB instances.

Inter‑service communication was performed via HTTP, with internal hosts mapped via the hosts file. Issues and optimizations included:

Replacing the unmaintained Nginx Push Module with a Tornado + ZeroMQ based service (tormq) for push notifications.

Lack of retry, error handling, and rate‑limiting in HTTP calls caused availability problems.

Frequent Nginx configuration changes made service management error‑prone.

Authentication logic was duplicated across services, requiring coordinated releases.

3. Microservices Architecture

The entry layer adopted Kong (built on OpenResty) as an API gateway, with custom plugins for authentication and rate limiting. Service registration was automated via Kong’s admin API during deployment.

A custom RPC framework named doge was built on gevent + msgpack, using etcd for service discovery and providing client‑side flow control, high availability, and load balancing.

Technical selection challenges included learning OpenResty/Lua and Golang, implementing a short‑URL service, and deploying Kong in a two‑node cluster behind a cloud load balancer.

The team also maintained a pure‑Python Thrift implementation ( thriftpy) but, due to limited manpower, ultimately created the doge RPC framework inspired by Weibo’s Motan.

4. Domain‑Driven Design

In the latest stage, the architecture introduced a data‑service layer where each bounded context exposes a single aggregate root via RPC. Application services depend on these data services, achieving decoupling and allowing higher‑level services to rely only on lower‑level ones.

At the time of leaving the company, DDD was still being studied and not yet fully implemented, but the author expects continued evolution toward this direction.

Conclusion

Architecture and technology choices should serve product and customer needs rather than follow trends blindly; compromises are inevitable, and finding optimal solutions within constraints is the biggest challenge. Emerging patterns like Service Mesh are gaining traction, and the author plans to keep learning about them.