Independent Deployment ≠ Independent Evolution: Designing Dependencies, Routing & Release Architecture
The article explains why independent deployment doesn't guarantee independent service evolution, detailing five required capabilities—identifiable dependencies, compatible interfaces, controllable traffic routing, verifiable releases, and recoverable failures—using a refund service case study to illustrate dependency topology, interface evolution patterns, phased release strategies, and evidence-based rollback decisions.
The article opens with a concrete scenario: a refund service has its own repository, database, and deployment unit, yet adding a new "pending manual confirmation" status forces four systems (order, message consumers, finance batch) to coordinate a simultaneous, ordered release. If one team isn't ready, no one can deploy—proving the services are independently deployable but not independently evolvable.
Independent Evolution Requires Five Capabilities
True independent evolution means a provider can release a compatible version first, consumers migrate on their own schedule, old and new versions coexist temporarily, and the process can be halted or rolled back. The author formalizes this as:
Independent Evolution =
Clear Dependencies
+ Compatible Interface Changes
+ Controllable Traffic Switching
+ Verifiable Release Process
+ Executable Exit & Recovery PlansTwo candidate release strategies are compared:
Simple release with fast rollback – suitable when consumers are few, changes are compatible, and business risk is low. Benefit: short path, low build/maintenance cost. Risk: hidden dependencies may make rollback insufficient.
Phased release with compatible migration – needed when consumers are many, changes cross teams, and external side effects are significant. Benefit: gradual verification, limited blast radius. Cost: requires maintaining routing, contracts, observability, and migration evidence.
The choice depends on change risk, dependency complexity, and the team's ability to sustain the mechanism—not on whether the system is labeled "microservices."
Map Dependency Topology, Not Just Call Arrows
Typical architecture diagrams only show request direction (e.g., order → refund → payment). For release decisions, five dependency types must be explicit:
Call dependencies – synchronous requests/responses must stay compatible across versions.
Data dependencies – consumers reading refund tables, cache keys, or internal structures directly; schema changes can break them without API involvement.
Message dependencies – notification and reconciliation consumers subscribing to refund results; old consumers may run long after new messages appear.
Batch dependencies – finance jobs reading refund data on a fixed schedule; online validation passing doesn't guarantee the next batch run succeeds.
Runtime dependencies – service discovery, gateway, config, certificates, keys; environment changes can break a release even if code is unchanged.
The dependency catalog must record provider/consumer, interface/message used, compatibility promise, failure owner, and deprecation confirmation criteria. Relying on chat queries or code search to find callers means dependencies are ungoverned. Dynamic calls, cron jobs, external clients, and ad-hoc scripts often live outside the current repo.
Sync, Message, and Batch Have Different Compatibility Windows
Synchronous calls – rolling deployments create mixed version pairs; must verify real version combinations, timeout budgets, and failure handling during the window.
Message collaboration – brokers retain old and new messages; consumers lag; schema evolution must be validated for unknown fields/enums and consumer migration status.
Batch processing – long execution cycles, late failure detection; must verify file/interface contracts and ensure observation windows cover critical business periods.
Cross-service state consistency, idempotency, and partial-failure compensation are deferred to a follow-up article.
Interface Evolution: Extend, Migrate, Shrink — Not Just Version Numbers
For the new enum value, the dangerous approach is returning it immediately and demanding all callers upgrade simultaneously. The safer pattern:
Extend – provider makes backward-compatible changes first: add optional fields, keep old fields, allow old and new requests to coexist. Even adding an enum value can break strict deserializers, so the contract must define unknown-value handling, not just JSON parseability.
Migrate – consumers adapt at their own pace. Contract tests verify known request/response combos; consumer-driven contracts reveal actual field and semantic dependencies. Tests only cover registered consumers; unregistered scripts and external callers remain invisible.
Shrink – removing old code isn't enough. Combine gateway access logs, message consumption records, batch execution logs, and owner sign-off to prove zero traffic during an agreed observation window. If consumers can't be fully identified, deprecation must pause, not proceed on schedule.
Versioned endpoints (/v2) have value for major semantic breaks that can't coexist in one contract, but they are not a default for every field tweak. Adding versions without designing consumer migration, traffic observation, and v1 exit criteria only leads to long-term version sprawl.
Routing Controls Blast Radius, Not Just Forwarding
Service discovery answers "which instances can receive requests"; routing answers "which instance should handle this request." Together they govern how compatible versions meet real traffic. A basic flow:
New version registers as candidate.
Health checks confirm readiness (not just liveness).
Routes send a small, identifiable traffic slice.
Observability gates approve gradual expansion; anomalies trigger stop and cutback.
Three common boundary confusions:
Process alive ≠ service ready. A hardcoded 200 OK liveness probe only proves the probe endpoint is reachable, not that DB connections, config loading, or critical dependencies are satisfied. Liveness decides restart; readiness decides routing entry.
Gateway/Mesh executes routing but cannot decide release success. Platforms can shift by version, percentage, region, or request tags, but business metrics and stop conditions must be defined jointly by the service team and business owner.
Traffic percentage is not a fixed constant. Low-risk reads and fund-refund writes need different canary scopes, observation durations, and stop criteria—driven by traffic profile, business cycle, failure impact, and recovery capability.
Timeouts, Retries, Circuit Breakers, Rate Limits — Assign Explicit Ownership
A frequent failure mode: gateway, client library, and business code all configure retries, or each layer assumes the next one handles rate limiting. A single payment-channel timeout can amplify into a storm of duplicate requests. The article assigns responsibilities:
Provider – publishes capacity limits, interface semantics, rate-limit responses, and idempotency guarantees.
Caller – sets call timeout, decides retry/fallback/block/escalate-to-human on failure.
Platform team – supplies service discovery, routing, unified observability, configurable traffic control.
Architecture/governance – sets default guardrails, reviews high-risk dependencies and cross-layer mechanism conflicts.
Business owner – accepts residual risk, decides handling of high-stakes outcomes (money, permissions).
Callers know their wait tolerance and failure meaning; providers must expose capacity and error semantics. For external side effects (refund, charge), network timeout must not trigger automatic retry. When the result is unknown, automation should stop creating side effects and enter query/reconciliation/human confirmation. Detailed fault models, circuit breaking, isolation, and recovery are reserved for later articles; here the key is that every layer's runtime mechanism must have a clear, non-conflicting ownership boundary, otherwise it's impossible to know which layer's behavior changed during a release.
Release Pipeline Verifies Candidates, Not Just Moves Artifacts
Each stage must answer a distinct question using the same immutable artifact; environment differences are expressed through controlled config. Otherwise pre-production evidence loses relevance.
Build & Test – code, static analysis, unit/integration tests pass. Cannot prove real dependencies and production traffic work.
Pre-production – candidate artifact, config, DB changes, and key paths collaborate. Cannot prove production-scale and real-traffic risk are acceptable.
Small-traffic verification – technical and business metrics under partial real traffic. Cannot prove long-term stability for uncovered business cycles.
Phased production – overall metrics, dependencies, and resources stay within bounds after expansion. Cannot replace post-release continuous observation and operational ownership.
Automated gates should check more than "tests green": contract compatibility (API + message), DB migration pre-checks, candidate error rate/latency/resource thresholds, business metrics (refund applications, channel submissions, unknown results), and rollback readiness (old version, config, steps). Only when these evidences are tied to the same version, config baseline, and release batch can the team answer "why we can expand."
Version Rollback ≠ Business Outcome Reversal
Rollback is often treated as a universal safety net, but it only works for certain changes. If the release performed destructive DB migrations, emitted new-format messages, called payment channels, or mutated external state, redeploying the old version does not automatically undo those facts. Therefore, pre-release must distinguish:
Version rollback – reverts software behavior.
Business recovery – compensates for executed side effects.
DB changes should follow expand-migrate-shrink so old and new apps work during the window; message formats must retain a compatibility period for lagging consumers; external side effects need business idempotency, query, and reconciliation. Irreversible changes require forward-fix designs upfront. Automation must have stop boundaries: reversible software errors with clear evidence → auto stop expansion and rollback; unknown payment results, irreversible migrations, unclear consumer scope, or uncertain post-rollback state → freeze, preserve scene, escalate to human. Auto-rollback cannot be packaged as a universal answer.
Release Conclusions Must Trace Back to Evidence
For the refund service, the article maps each architectural judgment to concrete verification targets and evidence sources:
Dependencies identifiable – known callers, message consumers, batch jobs, runtime deps have owners. Evidence: dependency catalog, traffic records, service directory.
Interfaces compatibly evolvable – version combos and unknown-value handling meet agreements. Evidence: contract tests, compatibility tests, consumer confirmations.
Routing limits impact – candidate versions identifiable, divertible, stoppable, revertible. Evidence: routing config, release drills, change logs.
Releases phased with gates – every expansion has explicit gate and metric basis. Evidence: pipeline records, metrics, logs, traces.
Old endpoints safely retired – zero residual requests in observation window, owner sign-off. Evidence: access logs, consumption logs, deprecation checklist.
Failures actionable – both rollback and business recovery paths validated. Evidence: rollback drills, runbooks, recovery records.
Passing tests, successful candidate deployment, and normal small-traffic metrics only prove covered behaviors, target environment startup, and observed traffic within bounds. Verifying the agreed business cycle, dependency scope, and recovery capability forms the higher-level release conclusion.
Don't Over-Engineer for Canary's Sake
Low-risk internal services with few callers, compatible changes, and fast rollback need only rolling deploy, health checks, contract validation, and clear owners. Signals of over-design: maintaining many routing rules for few services/low frequency; designing many canary stages without business metrics; buying traffic-governance products before having a dependency catalog. Conversely, when changes require multi-team lockstep, old interfaces linger for fear of deletion, and releases rely on manual chat monitoring, the "simple" approach is no longer simple. Adding compatible migration, phased release, and evidence gates then reduces collaboration and change risk.
Summary
Independent deployment means a service has its own runtime unit. Independent evolution further demands: dependencies identifiable, interfaces compatible, traffic controllable, releases verifiable, failures manageable.
Don't require all services to change at once;
instead, allow providers and consumers to evolve separately within a compatibility window.Platform teams provide and maintain traffic switching and gating capabilities; service teams own interface semantics and runtime outcomes; architecture/governance sets standards and cross-service constraints; business owners decide high-risk calls. Only when design, platform, and accountability align does independent deployment translate into true independent evolution.
The next article tackles the harder distributed-systems layer: how to handle state consistency, idempotency, async tasks, and partial failure when a refund spans order, refund, and payment boundaries.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Data Bricklaying Diary
Records practices, thoughts, and pitfalls on the data grunt-work journey, sharing content on data platforms, data analysis, data processing, data governance, knowledge graphs, and more. Less theory, more hands‑on, making complex data technologies simple.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
