How 7 Engineers Replaced Git with Scion for AI-Scale Monorepos in 5 Months

Background: Git's Limits at AI Scale

Anthropic's monorepo had outgrown Git's decentralized design. The main branch history approached 100 GB (25 GB compressed), and CI systems alone performed ~25 million checkouts daily. Fetching full data for each checkout would generate over 600 PB of daily transfer. Traditional mitigations — shallow clones, sparse checkouts, promisor remotes, hundreds of internal Git mirrors, and object-storage snapshots — became increasingly fragile as merge volume surged from ~2,500 to >30,000 per day in four months (12× growth).

The team realized they effectively used Git as a centralized system: all work centered on one authoritative repo, and clients (mostly AI agents and automation) were always online. Git's offline/distributed features were now bottlenecks.

Scion Design Principles

Scion was built around six core principles:

Clients always online. No local full repo; clients fetch only needed data from servers.

Client/server separation. Servers handle history queries, blame, commit creation, merges, rebases. Clients maintain a bounded local object cache, write working-copy files, watch filesystem changes, and expose a user-facing CLI.

Full source-control API. All query capabilities (ancestry, blame, merge-base, blob fetch) are exposed as RPCs, eliminating the need for tools to maintain local repos.

Horizontal scaling everywhere. Repository data is distributed; no single server holds a complete materialized repo. Components scale independently without packfile maintenance.

Lightweight, incremental, lock-free commits. Commit cost proportional to change size, not repo size. Ref updates use a single Spanner compare-and-swap, decoupled from changeset creation.

Sparse and virtual checkouts by default. CI tasks often need only diffs or metadata; they query APIs directly. When files are required, clients fetch only accessed directories (sparse) and load blobs on demand (virtual).

Data Structures: Changesets and Derived Data

The fundamental unit is a Changeset — a write-ahead log record storing the list of changed files relative to its first parent and content-addressed pointers to compressed blobs. Unlike Git commits (which point to a tree), changesets record deltas, enabling high-throughput ingestion without packfiles, locks, or cross-server coordination.

To answer historical queries (blame, file history, commit-graph traversal), Scion builds Derived Data structures recursively: each changeset's derived data is computed from its parents'. This framework allows adding new indexes (e.g., copy detection, path search) independently, choosing eager vs. lazy computation per index, and evolving server-side structures without client changes.

Architecture: Three-Tier Caching

Scion's fetch protocol differs from Git's negotiation: clients request desired tree and blob objects directly, recursing from the root tree. Every request targets an immutable content-addressed object, so any cache layer can satisfy it. The stack: scion-daemon — per-node cache serving all tasks on a compute node; loads a periodic snapshot at startup. scion-cache — regional in-memory cache servers transmitting objects in the exact on-disk format; on miss, forward to scion-server. scion-server — authoritative layer reading from Spanner or cloud storage.

All layers share the same RPC interface, but access policies are strictly enforced to isolate failures (e.g., a cache outage doesn't flood the authoritative tier). Cache hit rates exceed 99% at every level.

Migration and AI-Assisted Development

The 7-person team built Scion, the Antique code-review tool, and ~half a dozen supporting systems in 5 months while daily merges grew 10×. Early prototypes were hand-written; later, an internal preview of Claude Mythos rewrote the Git-to-Scion import pipeline unprompted, delivering 20× lower memory use and 4× speedup while preserving all implicit constraints.

Engineers shifted from line-by-line coding to architectural governance: defining system boundaries, rejecting AI proposals that violated core invariants (e.g., linear-scan vs. indexed query), and curbing over-engineering from adversarial reviews. They still review every change but focus on intent rather than syntax.

Measured Outcomes

CPU/memory footprint reduced 200× vs. Git mirror fleet.

Checkout latency dropped two orders of magnitude (minutes → seconds).

CI cluster regained 25% CPU capacity .

Zero repo-maintenance tasks (no packfiles, no gc).

Operational Innovations

Per-directory access control and semantic review policies (trivial vs. sensitive changes) secure the monorepo.

Superfan agents — dedicated Claude Tag instances monitoring telemetry/logs for scion-daemon, scion-cache, and Antique merge queues — detect subtle anomalies missed by traditional alerts.

Post-merge testing replaces global merge locks; conflicts are auto-resolved by agents before humans notice.

Broader Implications

Scion is not a general Git replacement; it targets massive monorepos with thousands of engineers or heavy agent traffic. Anthropic shares its design to inspire incremental Git improvements: server-side RPC APIs, standardized cross-host interfaces, enhanced sparse/virtual checkouts, and decoupled server architectures. The case demonstrates that small, AI-augmented teams can now build and migrate core infrastructure in months rather than years.