Who Controls Traffic After Splitting Services? A Guide to Service Governance

Splitting Microservices Introduces Governance Needs

In a monolith, modules share a single process, calls are local, and deployment is usually a single unit. After decomposing into independent services, many services must cooperate (e.g., order service calls inventory, payment, and risk‑control services; logistics service receives order events). This creates new challenges: services must locate each other, user requests need a single entry point, traffic must be distributed across instances, failures must be isolated, and peak traffic must be controlled.

Service Registration and Discovery

When a service starts, it registers its name and address (e.g., 10.0.0.8:8080) with a registry. Multiple instances register their own addresses (e.g., 10.0.0.9:8080, 10.0.0.10:8080). Callers query the registry to obtain the current list of healthy instances instead of hard‑coding IPs. Common registries are Eureka, Nacos, Consul, and Zookeeper.

Register: service starts and records itself in the registry.

Discover: callers look up the target service address from the registry.

API Gateway as Unified External Entry

Without a gateway, clients would need to know the address, authentication, authorization, and composition logic of every downstream service. An API gateway sits in front of the microservice cluster and handles:

Routing – forwarding requests to the correct service.

Authentication – verifying user login.

Authorization – checking user permissions.

Rate limiting – blocking excessive traffic.

Aggregation – combining results from multiple services into a single response.

Example: a front‑end request for order details is sent only to the gateway, which may call order, product, logistics, and payment services, aggregate their responses, and return the combined data.

Load Balancing: Selecting an Instance

When a service has multiple instances (e.g., three inventory services), load balancing distributes incoming requests among them to avoid overloading a single instance.

Typical algorithms:

Round‑Robin – A, B, C in turn.

Weighted Round‑Robin – stronger machines receive more requests.

Least Connections – the currently least‑busy instance gets the request.

IP Hash – the same client IP tends to hit the same instance.

Consistent Hash – minimizes data movement during scaling.

Choosing an algorithm depends on the scenario: similar machine capacity → round‑robin; differing capacity → weighted; varying request latency → least connections; session affinity needed → IP hash; distributed cache or sharding → consistent hash.

Circuit Breaker: Preventing Fault Propagation

If a downstream service (e.g., payment) repeatedly times out, continued calls tie up threads, increase request backlog, and eventually drag the upstream service (order) down—a phenomenon called fault propagation.

A circuit breaker monitors failure rates. When failures exceed a threshold, it opens the circuit and immediately returns a fallback response (e.g., “Payment service busy, please retry later”). After a cool‑down period it moves to half‑open, allowing a few trial calls to see if the downstream has recovered, then closes again if successful.

States:

Closed – normal calls.

Open – failures exceed threshold; calls are rejected.

Half‑Open – limited trial calls to test recovery.

Rate Limiting: Controlling Incoming Traffic Volume

Rate limiting caps the number of requests entering the system. If the system can handle 10,000 QPS but a promotion generates 100,000 QPS, allowing all traffic would overwhelm services, databases, and caches, causing a system‑wide avalanche.

The goal is to protect the system: excess requests are queued, rejected, or asked to retry later, ensuring that core flows remain functional. In a flash‑sale scenario, only a portion of requests are allowed to proceed to the order process; excess requests receive a “high demand, try later” message.

Degradation: Preserving Core Functions

When resources are scarce during peak traffic, the system should guarantee essential capabilities—browsing, placing orders, and payment—while temporarily disabling or simplifying secondary features such as comments, recommendations, points, complex reports, and personalized sorting.

Example: a product detail page may be reduced to showing only product info, stock, and the “Buy” button, postponing comments and recommendations until load eases.

Six‑Question Checklist for Service Governance

Where is the service? – Registration & discovery.

Where do external requests enter? – API gateway.

To which instance does a request go? – Load balancing.

What if traffic is too high? – Rate limiting.

What if a downstream fails? – Circuit breaker.

Who to keep, who to drop? – Degradation.