From Monolith to Microservices: A Real‑World Journey and Lessons Learned

Initial Requirements

A few years ago, Xiao Ming and Xiao Pi started an online supermarket. Their needs were simple: a public website for browsing and buying products, and an admin backend for managing users, products, and orders.

Website

User registration and login

Product display

Order placement

Admin backend

User management

Product management

Order management

The initial monolithic implementation was quickly built and deployed to a cloud server.

As Business Grows

Rapid competition forced the team to add promotions, mobile channels, and data‑driven personalization. Development pressure led to ad‑hoc extensions: promotion management and data analysis were crammed into the admin backend, while a new mobile app was built separately.

This resulted in duplicated business logic, tangled API calls, oversized services, shared database bottlenecks, and difficult deployment and testing.

Time for Change

Realizing the architectural flaws, Xiao Ming and Xiao Hong abstracted common capabilities into independent services: User Service, Product Service, Promotion Service, Order Service, and Data‑Analysis Service. Each application now consumes these services, eliminating most redundant code.

Although the database remained shared, the new service layer reduced coupling.

No Silver Bullet

During a shopping festival, a traffic surge caused the Promotion Service to fail, triggering a cascade of failures across the system. The team manually added new instances to recover, but the incident highlighted the need for better fault detection and isolation.

Monitoring – Detecting Fault Signs

To spot problems early, each component exposes a metrics endpoint. A collector (e.g., Prometheus) scrapes these metrics, stores them, and a UI (e.g., Grafana) visualizes them and sends alerts.

Tracing – Locating Issues

Distributed tracing records a traceId, spanId, parentId, requestTime, and responseTime for each request. The collected spans are stored in a system like Zipkin, allowing engineers to view call trees and pinpoint failing services.

Log Analysis

When log volume grows, a search‑engine approach is needed. The ELK stack (Elasticsearch, Logstash, Kibana) indexes logs, provides aggregation, and offers a UI for querying.

Gateway – Access Control and Service Governance

A gateway sits between callers and services, performing authentication, authorization, and routing. The team chose a coarse‑grained approach: one gateway for all external traffic, internal calls remain direct.

Service Registration and Discovery – Dynamic Scaling

Services register themselves with a discovery system (e.g., Consul, Eureka, etcd). Clients fetch the list of healthy instances, enabling automatic load balancing and seamless scaling.

Circuit Breaker, Service Degradation, Rate Limiting

If a downstream service becomes unresponsive, a circuit breaker opens to return errors quickly, preventing request pile‑up. Non‑critical services can be degraded to keep core functionality alive. Rate limiting protects services from overload and cascading failures.

Testing

Testing is organized into three layers: end‑to‑end tests for critical user flows, service‑level tests (with mock servers) for API contracts, and unit tests for individual code units.

Microservice Framework

To avoid repetitive integration code, the team built a lightweight framework that provides metric exposure, tracing injection, log forwarding, service registration, and routing. However, framework upgrades require coordinated releases across all services.

Service Mesh

Alternatively, a sidecar proxy (e.g., Envoy) can handle cross‑cutting concerns without modifying application code. The data plane (sidecars) handles traffic, while the control plane manages configuration. This approach reduces code intrusion but adds network overhead.

Conclusion

Microservices are not the final destination; future directions include Serverless, FaaS, and even a return to monoliths for certain scenarios. The team’s migration is complete for now, and they look forward to a well‑earned coffee break.