From Ad‑hoc to Strict Layering: Managing Service Dependencies at Ten‑Million QPS

An unexpected restart of an edge user‑tag service caused a core transaction chain to fail, exposing how reverse and circular dependencies can turn a clean layered architecture into a tangled web; the article explains the hidden costs of such chaos and outlines two hard rules and a step‑by‑step path to enforce strict, acyclic layering for systems handling tens of millions of QPS.

Random Bulletin
Random Bulletin
Random Bulletin
From Ad‑hoc to Strict Layering: Managing Service Dependencies at Ten‑Million QPS

How Dependencies Turn from Clear to Chaotic

Initially, services were few and dependencies were obvious, with a clear hierarchy: gateway at the top, business services in the middle, and infrastructure services at the bottom. As teams grew and requirements piled up, developers started calling any convenient service, even across layers, leading to a spider‑web of dependencies.

Root cause: a routine restart of an edge user‑tag service caused the entire transaction chain to collapse because the tag service both depended on and was depended on by core services, creating a reverse and circular dependency.

The Real Cost of Messy Dependencies

Fault propagation: In a clean layered system, failures are bounded to the layer that crashes. In a tangled system, a minor glitch in an edge service can travel through hidden paths and bring down core services.

Deadlock risk: Circular dependencies cause services to wait on each other, preventing independent startup and leading to "chicken‑egg" deployment deadlocks.

Cognitive and iteration cost: When the dependency graph is opaque, any change becomes a gamble; teams become reluctant to modify code, technical debt grows, and delivery slows.

Why Dependency Governance Changes at Ten‑Million QPS

At million‑QPS scale, dependency chaos is a nuisance that can be mitigated with occasional fallbacks. At ten‑million QPS, every stray dependency becomes a potential single point of failure because the traffic amplified tenfold can exhaust upstream resources instantly.

Precise degradation also hinges on clear layering: only when non‑core dependencies are cleanly separated can they be disabled without breaking core logic.

What Strict Layering Actually Means

Rule 1 – One‑way dependency: Services are organized into layers, and calls may only flow from higher to lower layers. This creates a directed acyclic graph (DAG), enabling deterministic startup order, deployment, capacity planning, and fault isolation.

Rule 2 – Stability principle: The more a service is depended upon, the less it should depend on others. Core infrastructure services (e.g., user, inventory) should be stable and provide capabilities without pulling in higher‑level services. Upper‑level services can change freely because few depend on them.

If a low‑level service needs upper‑level data, it should use dependency inversion or asynchronous events rather than synchronous reverse calls, thereby preserving the direction of dependencies.

Step‑by‑Step Path from Ad‑hoc to Strict Layering

Step 1 – Observe the current state: Build full‑stack tracing and service‑dependency topology to visualize real dependencies; invisible dependencies cannot be managed.

Step 2 – Classify services into layers: Assign each service to an access, business, or infrastructure layer. Violating edges (bottom‑to‑top calls or cycles) become obvious targets for cleanup.

Step 3 – Break cycles and invert dependencies: For each violating edge, decide whether to delete it, convert it to an asynchronous event, or introduce an abstract interface so the lower layer depends on an abstraction rather than a concrete upper service.

Step 4 – Institutionalize guardrails: Embed layering rules into code review and automated checks to prevent new reverse or cross‑layer dependencies from re‑emerging.

Strict Layering Is Not About Over‑restriction

Excessively rigid rules can cause unnecessary indirection; a direct down‑ward call that does not create a cycle is acceptable. The goal is to control dependency direction, not to ban every shortcut.

Each mitigation technique (asynchrony, dependency inversion, strict admission checks) trades one complexity for another; there is no free lunch. Effective governance requires judging which dependencies truly need fixing and whether the cost is justified.

Dependency Governance as the Invisible Foundation

From million to ten‑million QPS, dependency governance remains essential. It does not provide immediate performance gains like caching, but when a major incident occurs, the offending edge is almost always an unexpected dependency edge.

Maintaining a single‑direction, acyclic dependency graph is an engineering discipline that demands daily restraint against ad‑hoc calls, vigilant monitoring for reverse and circular links, and long‑term commitment to the rules. Only then can a system reliably sustain ten‑million‑QPS traffic.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

microserviceslayered architecturecircular dependencyHigh QPSservice dependencydependency governancereverse dependency
Random Bulletin
Written by

Random Bulletin

17-year internet software developer specializing in AI applications, networking, architecture, and open source. Led the delivery of network services handling hundreds of millions of concurrent devices and tens of millions of QPS, and has three years of experience designing and building an agent platform. Follow to stay updated.

0 followers
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.