GitHub Rebuilds Git for Agent-Scale Dev, Hits 35x Write Throughput

Background and Motivation

GitHub engineering blog published "Building Git infrastructure for agent-scale development" by Brian Celenza. GitHub is rebuilding its Git infrastructure while the service remains online to support "agent-scale" software development loads. As developers and AI agents work concurrently in the same repositories, the existing Git architecture faces new pressure requiring a different architectural design.

Scale of the Problem

Data from September 2025 to August 2026 shows Git activity on GitHub surged from 218.2 billion events per month to 473.3 billion, more than double. In September 2026 alone, developers and agents generated 7.38 billion commits (5x year-over-year), pushes grew 4.9x from 690 million to 3.35 billion per month, pull request merges nearly quadrupled, and GitHub Actions ran 3.26 billion times (4x increase). The busiest repositories received about one billion requests in August 2026. GitHub attributes this primarily to large engineering teams with busy CI pipelines and increasing AI agents.

Why AI Agents Create Pressure

Agents operate in tight loops, committing or checking out after almost every operation, making sub-millisecond latency a bottleneck. CI jobs clone or fetch the same branch tip thousands of times per minute. GitHub Actions ran 3.26 billion times in September, over four times the previous year.

Current Architecture: Spokes

The current Spokes system stores each repository on local disks of five file servers. A three-phase commit protocol with quorum ensures CI, web UI, and API clients see consistent state. This architecture serves one billion repositories.

However, every replica participates in every write, so push speed depends on the slowest replica. GitHub summarizes: "Adding replicas to share read load slows down writes." Losing quorum halts writes entirely.

New Architecture: Storage-Compute Separation

The new design separates storage and compute layers. Authoritative repository data lives in Azure Blob Storage, handling durability and replication. Lightweight compute worker nodes cache data for read requests and scale with traffic. If a worker fails, the impact equals a cache miss.

Coordination is minimized: only reference updates require consensus; object storage, connectivity checks, and secret scanning run largely in parallel. Compaction and garbage collection move to separate worker processes away from hosts handling live Git requests.

Results and Guarantees

Internal benchmarks show the new design achieves up to 35x write throughput improvement. GitHub emphasizes that branch protection, required reviews, and audit logs remain functional. The reorganization has no maintenance windows; the platform continues normal operation.

Source: https://github.blog/news-insights/company-news/an-update-on-github-availability/