Essential Design Principles for Building Scalable Microservices

Microservices have several key points. Below is the Spring Cloud ecosystem diagram.

Design Point 1: API Gateway

During microservice implementation, frequent service splitting requires a unified entry for mobile apps to route requests transparently. An API gateway enables data aggregation at the gateway layer, reducing app power consumption and improving user experience. It also centralizes authentication and authorization, exposing only necessary external interfaces and allowing internal service calls without repeated auth checks, improving efficiency. Additionally, the gateway can enforce policies such as A/B testing, blue‑green deployments, pre‑release traffic routing, and, being stateless, can scale horizontally without becoming a bottleneck.

Design Point 2: Statelessness

Distinguishing between stateful and stateless applications is crucial for migration and horizontal scaling. Stateless services externalize session, file, and structured data to unified backend storage, leaving only business logic in the service. Stateful components like ZooKeeper, databases, and caches remain in concentrated clusters. Stateless parts enable cross‑datacenter deployment and elastic scaling, while stateful parts rely on their own high‑availability mechanisms. Even with stateless design, in‑memory data may be lost on process failure, so services need retry and idempotency mechanisms, leveraging service discovery to retry against another instance.

Design Point 3: Database Horizontal Scaling

Databases store state and are common bottlenecks. Distributed databases allow performance to increase linearly with added nodes. The architecture uses a primary‑replica RDS for zero‑loss failover, load‑balancing NLB with LVS/HAProxy/Keepalived, and a Query Server that can scale horizontally based on monitoring data, providing transparent failover to the business layer.

Design Point 4: Caching

In high‑concurrency scenarios, layered caching brings data closer to users, reducing latency and load on back‑end databases. Mobile apps should cache critical, frequently changing data locally. Static data can be cached via CDN with periodic refreshes. When CDN misses, a fallback to origin occurs, where an access‑layer cache can intercept most requests. Dynamic data can be cached locally or using distributed caches such as Memcached or Redis, and can be partially static‑ized to further reduce backend pressure.

Design Point 5: Service Splitting and Discovery

When systems become unmanageable due to rapid changes, large services are split into smaller, independent services. Benefits include independent development, independent deployment, targeted scaling of critical transaction paths, and easier degradation of non‑core features during peak loads. Service discovery mechanisms manage inter‑service relationships, providing automatic repair, association, load balancing, and fault‑tolerant switching.

Design Point 6: Service Orchestration and Elastic Scaling

After splitting, the proliferation of processes necessitates orchestration to codify deployment and manage dependencies, embodying “infrastructure as code.” Orchestration files stored in version control enable atomic updates, rollbacks, and traceability for hundreds of services.

Design Point 7: Unified Configuration Center

With many services, local configuration files become unmanageable. A centralized configuration center distributes configurations, handling immutable settings baked into container images, startup parameters via environment variables, and dynamic configurations for feature toggles, degradation, and other runtime controls.

Design Point 8: Unified Logging Center

Collecting logs from thousands of containers requires a centralized logging system with a common log format, enabling end‑to‑end transaction tracing by searching for identifiers such as transaction IDs.

Design Point 9: Circuit Breaking, Rate Limiting, and Degradation

Services must implement circuit breaking, rate limiting, and graceful degradation. Timeouts should return fallback data promptly. Overloaded downstream services trigger circuit breakers to prevent cascading failures. During high load, non‑critical functions can be degraded, and rate‑limiting ensures the system operates within tested capacity, providing user‑friendly messages like “system busy, please retry.”

Design Point 10: Comprehensive Monitoring

Complex systems need unified monitoring for health status and performance bottlenecks. Integrated alerting detects anomalies, while full‑stack monitoring during stress tests identifies bottlenecks, preserves execution traces, and guides optimization.