Mastering Service Governance: From Distributed Systems to Reliable RPC Frameworks

We often hear the term "service governance" in distributed development. Before understanding it, we must first understand distributed systems, RPC communication, and the problems RPC frameworks face.

1. Common anomalies in distributed systems

Server crashes, network anomalies, the three possible RPC outcomes (success, failure, timeout), and data loss are typical issues that must be addressed.

2. Replica classification

Replicas can be divided into service replicas (stateless) and data replicas (stateful). Data consistency is a major challenge, with strong, weak, and eventual consistency models.

3. Design principles of distributed systems

Key principles include heterogeneity, transparency, concurrency, scalability, fault independence, data consistency, and load balancing.

4. Metrics for distributed systems

Performance is measured by TPS (transactions per second), while availability and scalability assess the system's ability to handle failures and growth.

RPC Framework

RPC (Remote Procedure Call) enables a program to invoke functions on a remote server as if they were local, abstracting complexities such as load balancing, serialization, network retries, and timeouts.

The framework consists of three roles: client, server, and registry. The client handles serialization, connection pooling, load balancing, fault tolerance, timeout, and asynchronous calls. The server processes incoming packets, manages I/O and worker threads, and performs serialization. The registry provides service registration and discovery.

RPC Call Flow

An RPC call involves five components: client, client stub, server stub, service provider, and network transport (TCP or HTTP). The client stub packages request parameters and sends them to the server stub, which unpacks the request and invokes the local service.

Service Governance

As applications evolve from monoliths to microservices, managing service URLs becomes cumbersome, requiring a unified system—called a registry—to handle service registration, discovery, load balancing, traffic shaping, version compatibility, circuit breaking, degradation, and rate limiting.

Key components include:

Service: the basic unit providing functionality via network calls.

Registry: the "address book" mapping services to their endpoints.

Service registration & publishing: services register themselves and send heartbeats.

Service discovery: client-side or server-side discovery to locate service instances.

Traffic shaping: techniques to smooth request spikes.

Version compatibility: ensuring new versions can handle old data and protocols.

Circuit breaking: preventing cascading failures by halting calls to unhealthy services.

Degradation: gracefully reducing functionality under high load.

Rate limiting: controlling request rates to protect system stability.

Load balancing strategies: distributing traffic across multiple instances.