Skip to main content
ProductivityRyan Robson4 min read

Why I Built Sharding

Building larger things with Claude Code, I kept losing time to the same failure: two AI sessions would agree on an interface, then one would quietly drift from it. Sharding treats that forgetfulness as fixed and catches the drift mechanically.

I kept hitting the same wall building larger things with Claude Code, and it wasn't the model being dumb. It was the model being forgetful in a way I couldn't design around.

Here is the shape of it. I would agree on an interface with a session in one part of the codebase - an endpoint, a payload, an event name. Then I would open a fresh session somewhere else to build the thing that consumes it. And that session, working from its own slice of the world, would quietly build against a slightly different shape. Not wrong exactly. Just not the thing we agreed on. A field renamed. An enum narrowed. A status code that used to be there and now isn't.

Nobody noticed until integration, which is the worst possible time to notice. And the usual fix - stuff more context into every session, write longer and longer instructions begging it to remember the whole product - never held. The context window is finite and the product isn't. I was fighting the tool's nature instead of working with it.

The reframe

The thing that unlocked it: stop treating LLM inconsistency as a bug to hope away, and start treating it as a fixed property to build around.

Once I accepted that any single session is going to be individually forgetful, the question changed. It is no longer "how do I make the model remember the whole system?" It is "how do I make the shared truth external, small, and impossible to silently violate?"

We already solved a version of this problem for humans. It is called a type checker. A type error doesn't rely on a developer noticing that two modules disagree - it blocks the build, mechanically, every time. That is the standard I wanted for agreements between AI sessions.

The mechanism

Sharding builds a product as a set of deliberately separated components - "shards" - each developed in its own isolated Claude Code session. They are coupled only through a frozen, versioned contract, and integrated in phases where each phase has to produce a provably integrable result.

Three rules do all the work:

  • The contract is external and frozen. APIs, schemas, shared conventions - they live on disk, small and explicit, not in any session's memory. While a phase is open, the contract's version is frozen.
  • Drift is caught mechanically, not noticed. A deterministic engine extracts each shard's actual declared surface and diffs it against its slice of the contract. If a shard has drifted - on the side it provides or the side it consumes - the check fails and reports exactly where. No agent has to catch it.
  • Each shard is isolated. A session opened inside a shard can see only that shard's directory plus a read-only copy of the contract. It can't read its neighbors and it can't write the contract. Drift and change are the same event from opposite sides: a shard diverging from the contract is drift (illegal, blocked); the conductor bumping the contract version is change (legal, versioned, and it marks every shard stale until each one explicitly re-acknowledges the new version).

That last channel matters more than I expected. A shape diff catches a renamed field. It can't catch a narrowed rule - an enum that lost a value, a constraint that got tighter - because the shape still looks compatible. Versioned acknowledgement catches those: bump the contract, and every shard is stale until a human-in-the-loop session reviews what actually changed and signs off. The phase can't advance on unreviewed change.

Does it actually work?

The repo ships a two-shard demo that runs the real engine end to end: an orders shard that provides an Order API, and a gateway shard that consumes it. The end-to-end test drives the actual checker, not a mock. You can clone it and watch the graph go green - or introduce a drift and watch the gate stop it.

That is the thesis under test: N independently-driven, isolated shards, coupled only through a frozen contract and gated mechanically, snap together into an integrable deliverable at each phase - despite every session being individually forgetful.

Where it's at

It is an honest proof of concept. The plugin and the engine prove the mechanism; the obvious next step - an orchestrator that actively drives the shard sessions rather than you doing it by hand - is deliberately out of scope for now. It is source-available under BUSL 1.1 (individual use is explicitly granted, converts to MIT in 2030), and it installs as a Claude Code plugin or runs as a plain CLI in CI.

I am not trying to sell anything with this. I would just like people to use it and tell me where it breaks. If the idea resonates, or if you try it and it falls over, the issues are open:

github.com/robworks-code/sharding

More in Productivity