Microservice Architecture: Stability, Service Degradation, Data Consistency, and Migration Practices

Introduction

Microservices have become increasingly popular, offering modularity, reduced coupling, data isolation, clear business boundaries, and fewer code conflicts, but they also introduce new challenges such as system stability, complex service dependencies, data consistency, and operational overhead.

1. Ensuring Microservice System Stability

Snowball (avalanche) effect: a single service failure can cascade through the call chain, causing widespread outages.

Service call chains become longer, making fault location harder.

Data consistency issues arise when a request spans multiple services and databases.

Mitigation approaches include:

Adding circuit breakers between services (e.g., Hystrix, Resilience4j) to fail fast and protect downstream services.

Thread isolation within JVMs to avoid thread‑pool contamination.

Rate limiting at the gateway (Zuul, Gateway, Nginx) to handle traffic spikes and protect backend resources.

Cache strategies: handling cache penetration, cache avalanche, and cache hotspots.

Deployment, data, and business isolation to prevent a single high‑traffic scenario (e.g., flash sales) from affecting unrelated services.

Redundant data copies to provide fallback data during service failures.

Network bandwidth planning and graceful gray‑release mechanisms.

2. Service Degradation Techniques

Manually disabling non‑critical features during high load (e.g., hiding logistics queries during large promotions).

Read‑side degradation: serving data from cache only when the database is under pressure.

Write‑side degradation: converting synchronous calls to asynchronous message queues, sacrificing immediate consistency for throughput.

Data redundancy to allow services to continue operating when dependent services fail.

Circuit breaking with fallback data (e.g., product service fallback to locally cached price).

Rate limiting at the gateway and service level, with configurable thresholds and dynamic adjustments.

Classification of degradation into manual (switch‑based) and automatic (circuit breaker, rate limiting) approaches.

3. Data Consistency in Microservices

When a single request touches multiple services, traditional ACID transactions no longer work. The article explains two patterns:

TCC (Try‑Confirm‑Cancel) : reserve resources in the Try phase, then either Confirm to commit or Cancel to roll back. Example code snippets show how order status, inventory, and coupon services cooperate using TCC frameworks such as Hmily.

Message‑Based Final Consistency : use transactional messages (e.g., RocketMQ) where a half‑message is sent, the local transaction is executed, and then the broker commits or rolls back the message based on the transaction outcome. The broker also performs periodic checks to resolve uncertain states.

Both patterns require idempotent operations and handling of “suspension” scenarios where the second phase arrives after the first phase has already been rolled back.

4. Seamless Data Migration During Microservice Refactoring

Enable dual‑write to old and new databases; ensure all CRUD operations write to both.

Migrate historical data (up to a chosen timestamp) to the new schema using scripts, ignoring conflicts that are already handled by dual‑write.

Validate migrated data for completeness and correctness.

Introduce dual‑read, gradually shifting read traffic to the new database while falling back to the old one when needed.

After full migration, disable writes to the old database and clean up migration code.

5. Full‑Stack APM Monitoring

To address the operational complexity of microservices, the article demonstrates using open‑source APM tools such as Pinpoint (alternatives include SkyWalking, commercial SaaS solutions). Key features highlighted:

Topology graph visualizing service‑to‑service, service‑to‑database, and service‑to‑cache dependencies.

Call‑stack view showing per‑method latency and pinpointing bottlenecks (e.g., slow SQL queries).

Server map indicating request frequencies per node, helping identify hot spots.

JVM metrics (heap, GC, thread count, CPU, file descriptors) for resource monitoring.

These tools enable rapid diagnosis of performance issues and errors without manually combing through logs.

Conclusion

The article provides a comprehensive, experience‑driven guide to microservice adoption, covering architectural benefits, stability safeguards, degradation strategies, consistency mechanisms, data migration practices, and monitoring techniques, offering practical insights for engineers undertaking microservice transformations.