GitHub Unveils Enhanced Git Storage Architecture for High-Volume Development

GitHub has introduced a new Git storage architecture designed to handle thousands of writes per second while maintaining low latency for read operations. This architecture supports up to 5,000 concurrent agents pushing 7,200 updates per second to a repository.

Announced during GitHub Universe, the architecture aims to facilitate agentic development for millions of developers. It was developed alongside GitHub’s ongoing operations, ensuring minimal disruption during the transition.

Over the past year, the rise of agents has transformed the dynamics of busy repositories on GitHub. In previous discussions, GitHub outlined two key principles for this architecture: minimizing coordination and decoupling storage from compute resources.

The architecture emphasizes that only reference updates require agreement, allowing various components of the Git infrastructure to scale independently. Developers will continue to interact with Git in the same manner, maintaining existing protections while benefiting from the new system.

Scaling for Agent Development

GitHub’s scaling strategy focuses on two dimensions: horizontal scale, which accommodates multiple developers across various repositories, and vertical scale, which supports many developers working on a single repository. Current high-traffic repositories experience dozens of writes per second, but as agent-driven development accelerates, GitHub anticipates a shift to sustained higher rates of activity.

As the demand for agent-driven development increases, GitHub aims to enhance its architecture to support thousands of writes per second and multiple writer nodes. This approach not only improves fault tolerance but also reduces latency for commits.

Project Rubicon

In May 2026, GitHub initiated the design of what it calls “Project Rubicon,” which aims to meet the demands of high-volume development. The architecture utilizes Azure Blob Storage for durable data and a new replicated log service, Ref-CAS, for managing references.

Requests from Git clients are routed through Spokes, which manages load balancing and routing while maintaining tenant fairness. Each repository’s traffic is directed to a small group of virtual replicas that serve read and write requests, ensuring low latency and efficient cache management.

When a repository experiences increased activity, additional virtual replicas can be added to manage the load. Background compaction nodes work separately from the virtual replicas to optimize data storage without impacting performance.

Optimizing Reference State Management

The architecture’s design allows for efficient management of reference states, which are critical for maintaining high write throughput. The Ref-CAS service enables conditional writes that only lock the references being updated, allowing for parallel processing of most updates.

Each reference update is committed through a quorum, ensuring that only one update to a reference is recorded at a time. This method allows GitHub to achieve tens of thousands of operations per second while minimizing the impact on read performance.

As GitHub continues to refine its new architecture, it aims to improve overall reliability and scalability, preparing for the future of agent-driven development.

Source: GitHub Blog