Designing High‑Availability Distributed Systems: SOA, Microservices & Service Mesh

In the previous article we discussed the evolution of distributed architectures; this article examines the current mainstream distributed architectures, common theories, and how to design highly available distributed systems.

SOA Architecture Overview

SOA (Service Oriented Architecture) is a design philosophy that composes multiple independent services which communicate over a network to provide complete functionality. Services are typically deployed independently.

Enterprise Service Bus (ESB) acts as a pipeline connecting service nodes, handling protocol integration, message transformation, and routing. As services grow, the call graph becomes complex; introducing ESB simplifies the relationships.

With ESB, the invocation diagram becomes clearer.

Core Problems SOA Solves

System integration: Transform chaotic mesh structures into orderly star topologies using concepts like ESB and service management standards.

Service orientation: Abstract business logic into reusable, composable services for rapid business regeneration.

Business serviceification: Convert enterprise functions into reusable services to enhance external service capability, focusing on efficiency.

Microservices Architecture Overview

Microservices are an evolution of SOA, emphasizing complete componentization and service‑orientation. A monolithic system is split into many small, independently developable, deployable applications that interact via services. Components are analogous to hardware parts (CPU, memory) that can be upgraded without affecting others. The service mesh (ESB) acts as the motherboard.

Characteristics of Microservices

Componentization through services

Division by business capability and development teams

Decentralization

Infrastructure automation (DevOps, automated deployment)

Differences Between SOA and Microservices

Microservices de‑emphasize the heavyweight ESB of traditional SOA and apply SOA principles within individual business systems to achieve true componentization. Container technologies like Docker enable small deployment units, allowing each service to run independently (e.g., Spring Boot, Node).

SOA focuses on system integration, while microservices prioritize complete isolation.

Service Mesh Architecture Overview

Service Mesh, a non‑intrusive infrastructure layer for service‑to‑service communication, handles networking, circuit breaking, rate limiting, and monitoring, abstracting these concerns away from application code.

Popular open‑source Service Mesh projects include Linkerd, Envoy, Istio, and the Kubernetes‑native Conduit.

Service Mesh Features

Intermediate layer for application communication

Lightweight network proxy

Application‑transparent

Decouples retries, timeouts, monitoring, tracing, and service discovery from the application

Fundamental Distributed System Theories

1. Distributed Consistency

Different services have varying consistency requirements (e.g., ticketing vs. bank transfers). Replication introduces latency, leading to temporary inconsistencies.

Strong consistency ensures reads reflect the latest writes but hurts performance; weak consistency relaxes immediacy; eventual consistency guarantees convergence over time.

2. CAP Theorem

In a distributed system, it is impossible to simultaneously guarantee Consistency, Availability, and Partition tolerance; at most two can be achieved.

Consistency: All nodes see the same data at the same time.

Availability: Every request receives a response, success or failure.

Partition tolerance: The system continues operating despite network partitions.

3. BASE Theory

BASE (Basically Available, Soft state, Eventually consistent) relaxes ACID constraints to improve availability and performance, especially for large‑scale systems.

Basically Available

System remains partially available during failures (e.g., degraded response times, reduced user coverage).

Soft State

Data may exist in intermediate states without affecting overall availability; synchronization delays are tolerated.

Eventually Consistent

All replicas converge to the same state after some time, even if not immediately.

High‑Availability Design for Distributed Architectures

Avoid Single Points of Failure

Load balancing (hardware, software, gossip‑based clusters)

Hot standby (Linux HA)

Multi‑datacenter deployment (same‑city and cross‑region disaster recovery)

Application‑Level High Availability

Failure monitoring and automatic alerts (system, link, log monitoring)

Fault‑tolerant design (service degradation, rate limiting)

Data scalability (sharding, read/write separation)

Scalable Design

Vertical scaling

Hardware upgrades

Horizontal scaling

Adding more servers

CDN for Accelerating Static Content

Content Delivery Network caches static resources near users, reducing bandwidth and latency.

DNS resolves the domain to a CDN‑specific DNS server.

CDN global load balancer returns the IP of the nearest edge node.

User requests content from the edge node.

Edge node selects an optimal cache server based on IP, content name, and load.

Cache server serves the content; if missing, it fetches from upstream caches until the origin is reached.

When to Use CDN

Ideal for static, infrequently changing assets such as images, JavaScript, CSS, and template background images.

Gray‑Release Deployment

Gradually roll out new versions to a subset of users, monitor performance, and roll back if issues arise, ensuring zero‑impact user experience.

Conclusion

This article introduced mainstream SOA, microservices, and service‑mesh architectures, explained fundamental distributed system theories, and outlined practical strategies for designing highly available distributed systems. The next article will demonstrate how to build a microservice system based on DDD.